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I.  INTRODUCTION 


Background  Information 

In  an  era  of  ever-increasing  emphasis  on  providing  efficient 
ambulatory  health  care  services,  it  becomes  extremely  important 
to  establish  an  effective  means  by  which  to  schedule  patient 
appointments.  Although  Army  Regulation  40-4  contained  references 
tc  the  use  of  a  Central  Appointment  System  (CAS)  as  early  as 
1967,  the  Department  of  the  Army's  search  for  the  optimal 
appointment  system  actually  began  in  1972,  when  the  Office  of  the 
Surgeon  General  (OTSG)  directed  the  Health  Care  Studies  Division, 
Academy  of  Health  Sciences,  to  prepare  a  protocol  for  studying 
the  advantages  and  disadvantages  of  a  CAS  versus  a  decentralized 
system.  This  project,  entitled  "A  Study  of  Appointment 
Scheduling  Control  for  Outpatients,"  was  completed  in  April, 

1972. 

The  OTSG  Health  Care  Research  Advisory  Board  approved  the 
protocol  in  July,  1972;  however,  it  directed  that  the  protocol  be 
modified  to  restrict  the  effort  to  determine  the  most  efficient 
and  most  effective  method  of  operating  a  CAS.  The  change  in  the 
protocol  was  apparently  made  because  the  OTSG  was  experiencing 
pressure  from  the  Department  of  the  Army  to  enforce  the  CAS 
requirement  which  had  been  placed  in  Army  Regulation  40-4  five 
years  earlier. ^ 
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During  this  same  time  period,  the  Comptroller  of  the  Army 
was  conducting  a  study  to  analyze  the  workload  of  outpatient 
clinics  to  determine  management  practices  which  might  be  useful 
in  improving  overall  efficiency.  As  a  result  of  a  recommendation 
from  this  study,  the  Chief  of  Staff  of  the  Army  directed  the  OTSG 
in  July,  1972,  to  notify  all  hospitals  that  appointment  systems 
were  to  be  standardized  and  centralized  under  the  Department  of 

Clinics. 2 

The  Health  Care  Studies  Division  completed  the  CAS  study  in 
January,  1973.  Since  the  decision  had  already  been  made  to 
designate  the  CAS  as  the  system  of  choice,  the  final  study 
results  did  not  have  to  defend  the  superiority  of  the  CAS  but 
simply  outlined  methods  to  be  used  in  implementing  or  upgrading 
an  existing  CAS. 

In  spite  of  the  increased  emphasis  on  the  CAS,  however, 
local  medical  treatment  facility  commanders  were  quite  resistant 
to  a  complete  CAS  implementation.  In  May,  1973,  and  again  in 
August,  197^,  Headquarters,  Department  of  the  Army  published 
letters  instructing  hospitals  to  comply  with  the  published 
directives  concerning  the  operation  of  a  CAS.  In  1975,  the  Army 
Audit  Agency  found  that  hospitals  continued  to  rely  on  the 
previously  established  decentralized  appointment  system,  which 
either  duplicated  or  assumed  CAS  workload.  The  Health  Services 
Command  Inspector  General  reported  that,  based  on  a  review  of 
inspection  reports,  numerous  hospitals  had  clinics  which  were 
operating  a  dual  appointment  system.  Interviews  conducted  by 
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inspectors  revealed  an  unwillingness  of  local  commanders  and 
health  care  providers  to  accept  the  concept  of  a  CAS . 3 

In  May,  1977,  the  OTSG  imposed  a  moratorium  on  the 
requirement  to  implement  a  complete  CAS.  In  addition,  the  OTSG 
allowed  the  local  commanders  to  determine  the  most  appropriate 
method  of  appointment  scheduling  in  order  to  provide  maximum 
patient  accessibility  to  the  appropriate  levels  of  care. 

The  recent  publication  of  HSC  Pamphlet  40-7-1  in  April  1984 
established  the  formation  of  the  Patient  Appointment  System  (PAS) 
which  is  a  combination  of  centralized  and  decentralized 
appointment  systems  within  each  treatment  facility.  Local 
commanders  are  now  authorized  to  modify  their  existing 
appointment  systems  to  establish  a  totally  centralized  or 
decentralized  system  or  a  PAS  consisting  of  a  combination  of  both 
types  of  systems. 


Development  of  the  Problem 

Madigan  Army  Medical  Center  (MAMC),  located  in  Tacoma, 
Washington,  adjacent  to  Fort  Lewis,  Washington,  is  a  U.S.  Army 
Health  Services  Command  (HSC)  medical  center.  MAMC  provides 
comprehensive  inpatient  and  outpatient  services  as  well  as 
veterinary  services,  environmental  health  services,  and  dental 
services  for  all  eligible  beneficiaries.  It  also  serves  as  a 
tertiary  care  center  for  the  Madigan  Army  Health  Services  Region 


to  Include  all  Department  of  the  Army  Medical  Department 


activities . 
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During  the  first  three  quarters  of  Fiscal  Year  1985  MAMC 
operated  an  average  of  378  beds  with  a  daily  average  bed 
occupancy  of  289.  Inpatient  admissions  averaged  56.6  per  day 
with  an  average  length  of  stay  of  5.1  days  and  average  daily  live 
births  of  7.3.  Outpatient  Services  include  thirty-seven 
specialty  clinics,  seven  troop  medical  clinios,  one  general 
outpatient  clinic,  and  one  cwenty-four  hour  emergency  room. 
Average  daily  outpatient  clinic  visits  for  the  first  three 
quarters  of  Fiscal  Year  1985  were  2714.^ 

In  1967  MAMC  first  established  a  central  appointment  system. 
Appointments  were  made  manually  by  the  appointment  clerks  and  the 
Information  was  keypunched  on  cards  and  fed  into  a  computer  to 
generate  clinic  schedules.  This  system  remained  in  effect,  with 
the  addition  of  a  rotary  wheel  in  December,  1975  and  an  automatic 
call  distribution  system  in  October,  1979.  The  system  was  fully 
automated  in  May,  1982  utilizing  a  Burroughs  B-I865  computer  with 
cathode  ray  tube  terminals  for  appointment  clerk  input.  The 
Burroughs  Patient  Appointment  System  was  established  as  a 
prototype  and  was  to  be  evaluated  during  a  two-year  period  to 
determine  if  the  system  was  adequate  to  complete  the  task  of 
making  appointments  efficiently.  In  addition  to  operating  the 
Patient  Appointment  System,  the  B-1865  computer  was  also  used  to 
operate  all  of  the  other  HSC  standard  automated  systems  and  the 
locally  established  operating  systems  for  MAMC.  In  June,  1984, 
due  to  the  lack  of  adequate  computer  support  by  the  existing 
Burroughs  computer,  the  MAMC  Commander  requested  an  upgrade  of 
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the  Burroughs  system  through  acquisition  of  a  Burroughs  B-1955 
computer  for  dedicated  use  by  the  Patient  Appointment  System. 

The  poorly  designed  software  of  the  Burroughs  Patient  Appointment 
System  requires  a  large  amount  of  memory  and  therefore  causes  the 
computer  to  be  tied  up  processing  information  for  the  Patient 
Appointment  System  while  other  systems  are  not  able  to  be 
processed.  The  Burroughs  1955  was  received  and  installed  in 
early  July,  1985.  The  addition  of  an  expanded  memory  capability 
by  early  fall  1985  should  provide  the  PAS  with  more  efficient 
automation  support. 

The  MAMC  PAS  has  continually  been  a  source  of  patient 
complaints.  In  August  1984  the  Health  Care  Consumer  Committee 
recommended  that  an  ad  hoc  committee  be  formed  to  study  the  PAS 
and  determine  what  problems  exist  which  limit  patient  access  and 
contribute  to  the  numerous  formal  complaints  (approximately 
twenty  -  thirty  per  month)  presented  to  the  Patient 
Representative  Officer.  This  committee  was  formed  in  September 
1984  and  consisted  of  six  members  of  the  MAMC  staff.  The 
committee  determined  that  there  were  three  major  deterrents  to  an 
efficient  PAS  at  MAMC.  The  issues  of  communication  support, 
automation  support,  and  personnel  staffing  were  identified  as 
major  problem  areas  of  the  PAS.  A  copy  of  the  Memorandum 
prepared  by  the  committee  for  the  Deputy  Commander  for  Clinical 
Services  can  be  found  at  Appendix  A. 

With  the  publication  of  a  newly  revised  HSC  Pamphlet  40-7-1 
the  command  group  of  MAMC  is  currently  evaluating  the 
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alternatives  of  (1)  converting  the  existing  system,  which  is  a 
combination  of  centralized  and  decentralized  modes,  to  a  totally 
centralized  or  decentralized  system  or  (2)  refining  the  existing 
system  to  more  adequately  meet  the  patients'  needs.  In  order  to 
adequately  assess  the  patients'  needs,  research  was  conducted  to 
determine  which  mode  of  appointment  system  is  the  most 
satisfactory  to  the  patient  population.  Even  though  consideration 
will  be  given  to  the  issues  of  cost  and  staff  desires  for  any 
recommended  changes  to  the  existing  system,  a  major  decision 
factor  is  reducing  the  number  of  patient  complaints  and  meeting 
the  needs  of  the  patient  population. 

Research  Question 

To  determine  if  there  is  a  significant  difference  in 
patients'  level  of  satisfaction  for  those  patients  utilizing  a 
centralized  versus  a  decentralized  system  for  scheduling 
outpatient  appointments  at  Madigan  Army  Medical  Center. 

Obj  ectives 

The  specific  objectives  of  this  research  were  to: 

1 .  Conduct  a  thorough  search  of  the  current  literature  to 
gather  information  which  is  relevant  to  the  research  topic. 

2.  Develop  a  suitable  questionnaire  for  use  as  the  survey 
instrument  for  the  research. 

3.  Conduct  a  pretest  of  the  survey  instrument  to  establish  the 
validity  of  the  survey  questionnaire. 
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4.  Determine  randomly  selected  patients'  levels  of  satisfaction 
with  the  scheduling  method  for  outpatient  appointment  (either 
centralized  or  decentralized)  at  MAMC,  utilizing  the  validated 
survey  instrument. 

5.  Analyze  the  data  gathered  from  the  survey  questionnaire, 
using  Chi-Square  and  T-Tests  to  examine  the  questionnaire 
responses,  both  individually  and  in  the  aggregate. 

6.  Provide  Information  to  the  Commander  on  the  results  of  the 
research  in  order  to  allow  him  to  evaluate  the  current  system  of 
making  outpatient  appointments  at  MAMC  and  to  initiate  changes  if 
Indicated . 


Criteria 

Criteria  for  the  research  was: 

1.  The  patients'  level  of  satisfaction  with  a  centralized  versus 
a  decentralized  appointment  system  was  compared,  utilizing  a 
statistical  significance  level  of  .05. 

2.  The  practical  significance  of  the  results  was  evaluated 
utilizing  the  prescribed  standard  of  Health  Services  Command  for 
patient  satisfaction.  This  standard  requires  an  eighty  percent 
response  of  satisfaction  in  order  to  denote  that  a  particular 
issue  is  a  patient  satlsfler.  Response  to  Individual  questions 
on  specific  aspects  of  the  appointment  system  as  well  as  the 
overall  opinion  score  were  evaluated  utilizing  this  eighty 
percent  standard.  Satisfaction  is  defined  as  any  value  greater 


than  or  equal  to 


0  on  a  five-point  satisfaction  scale. 
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3.  A  return  rate  of  seventy  percent  for  the  in-hospital  survey 
and  twenty  percent  for  the  mall-out  survey  was  required  in  order 
to  consider  the  survey  process  as  valid. 5 

Assumptions 

For  purposes  of  this  study,  it  was  assumed  that: 

1.  During  the  time  between  the  patients'  Interaction  with  the 
appointment  system  and  the  actual  administering  of  the  survey, 
some  loss  of  recall  of  information  may  have  occurred. 

2.  Patients'  perceptions  of  the  appointment  system  may  have  been 
biased  in  that  they  were  based  not  only  on  their  interaction  with 
the  military  facility  appointment  system  but  also  on  their 
immediate  and  past  experiences  in  health  care  settings,  including 
civilian  facilities. 

3.  Methods  of  validation  of  the  survey  instrument  for  this 
research  (use  of  an  expert  in  survey  design,  evaluation  by  a 
panel  of  individuals  with  previous  work  experience  with  the  MAMC 
appointment  system,  and  administration  of  a  pretest  to  a 
representative  sample  of  the  population)  were  sufficient. 

Limitations 

This  study  was  constrained  by  the  following  factors: 

1 .  Inability  of  the  researcher  to  be  physically  present  to 
administer  each  survey  to  the  patient  respondents  reduced  the 
number  of  completed  surveys  and  did  not  provide  the  respondents 
with  a  source  for  clarification  of  questions  about  the  survey. 
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2.  Based  on  the  physical  plant  layout  and  the  large  number  of 
clinics  at  MAMC,  coordination  to  administer  the  survey  on  the 
same  day  was  not  practical. 

Literature  Review 

Scheduling  is  the  determination  of  when  or  in  what 
order  individual  tasks  of  an  already-selected  set 
of  jobs  are  to  be  performed.  It  involves  allocating 
available  resources  to  specific  jobs  at  definite 
points  in  time  or  in  a  definite  sequence. 

Werner  F.  0.  Daechsel,  PhD^ 

The  earliest  article  found  relating  to  appointment  systems 
and  scheduling  was  written  in  1952.  In  their  article  Welch  and 
Bailey  discuss  appointment  scheduling  as  related  to  the  Issues  of 
punctuality  (both  for  patients  and  providers)  and  the  time  spent 
by  the  provider  in  consultation  with  each  patient.  Their  final 
conclusion  is  that  any  appointment  system  should  be  designed  with 
consideration  for  each  provider's  average  consultation  time  and 
the  premise  that  as  a  clinic  becomes  more  efficient,  the  patients 
and  providers  will  become  more  punctual. ^ 

A  review  of  the  recent  literature  published  on  the  subject 
of  appointment  systems  reveals  a  number  of  articles.  These 
articles  deal  with  several  different  aspects  of  appointment 
systems.  The  major  categories  into  which  these  articles  can  be 
classified  are  as  follows:  (1)  failed  appointments  or  no-shows; 
(2)  automation  considerations  in  appointment  systems;  (3) 
comparison  of  centralized  and  decentralized  appointment  systems; 
and  (4)  advantages  of  a  centralized  appointment  system.  Each  of 
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these  major  categories  will  be  discussed  as  they  relate  to  the 
reviewed  literature. 

Many  authors  within  the  past  ten  years  have  addressed  the 
major  issue  of  failed  appointments  or  no-shows.  Oppenheim, 
Bergman,  and  English  have  found  that  the  primary  reasons  for  this 
problem  are  a  lack  of  communication,  the  length  of  appointment 
interval,  the  absence  of  a  sense  of  urgency  for  keeping  the 
appointment,  and  the  lack  of  a  personal  physician.®  They  found 
that  no-show  rates  ranged  from  five  to  eleven  percent  in  family 
practice  centers,  and  nineteen  to  twenty-eight  percent  in  general 
outpatient  clinics. 9  Mailed  reminders  were  found  to  be  the  most 
cost-effective  intervention  in  four  studies . ^ ® ^ ^ ^ ^ ^ 3 

Several  articles  show  different  approaches  to  automation 
which  can  Increase  patient  and  provider  satisfaction  with 
appointment  systems.  Herpok,  Hansen,  and  Ritter  discuss  the  use 
of  the  Total  Medical  Record  (TMR)  system  employed  at  Duke 
Univesity  Medical  Center. This  automated  system  has  been 
designed  to  provide  an  autonomous  scheduling  system  which  matches 
patients  with  health  care  providers  and  therefore  efficiently 
allocates  scarce  resources. ^5  xhe  conclusions  of  the  article  are 
that  the  TMR  allows  providers  control  over  their  individual 
schedule  while  allowing  patients  immediate  access  with  up-to-the- 
minute  Information  about  the  complete  array  of  available 
appointments. 1 ®  A  different  approach  to  computerized  scheduling 
is  presented  by  Ratzer,  Fletcher,  Pollack,  and  Fletcher  in  their 
article  about  mini-computer  supported  appointment  systems. 
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Their  discussion  of  a  mini-computer  appointment  system  addresses 
the  issues  of  cost  reduction  the  ability  to  tailor  the  software 
tc  meet  the  needs  of  each  individual  clinic.  In  a  facility  which 
does  not  have  the  funds  available  to  purchase  a  mainframe 
computer,  the  use  of  mini-computers  for  appointment  scheduling 
may  be  the  best  solution. A  third  automated  system  discussed 
in  the  literature  is  the  Special  Computer  Applications  in  Medical 
Practice  ( SCAMP ) system .  Shapiro  provides  an  overview  of  the 
SCAMP  clinical  information  system  for  ambulatory  care. ^9  The 
four  basic  advantages  of  the  SCAMP  system  are  (1)  ease  of  use, 

(2)  comprehensiveness,  (3)  incorporation  of  medical  knowledge, 
and  (4)  ease  of  modification  of  system  capabilities. 20  The  SCAMP 
system  is  currently  in  use  at  the  Medical  University  of  South 
Carolina  and  both  the  physician  staff  and  patients  are  very 
pleased  with  the  simplicity  of  the  system  and  the  information  it 
provides .  2  "• 

Relsman,  Mello  da  Silva,  and  Mantell  conducted  an  extensive 
investigation  into  the  systems  and  procedures  for  outpatient 
flow. 22  They  address  the  distinct  advantages  of  both  centralized 
and  decentralized  appointment  systems.  In  the  centralized  system 
(1)  calls  for  appointments  are  always  correctly  directed;  (2) 
appointment  clerks  know  the  available  times  for  each  provider, 
allowing  for  easy  coordination  of  multiple  appointments;  (3) 
paperwork  is  kept  to  a  minimum;  and  (4)  economy  of  scale  must 
result. 23  jn  the  decentralized  system,  (1)  appointments  are  made 
for  only  a  few  providers,  usually  in  a  single  specialty;  (2)  the 
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orientation  period  for  appointment  clerks  can  be  shorter;  (3) 
follow'up  appointments  can  be  made  immediately;  and  (4)  providers 
can  check  and  adjust  their  individual  schedules. Although  this 
study  made  specific  recommendations,  the  important  conclusion  was 
that  for  any  system  to  function  properly  it  is  imperative  that 
each  individual  understands  how  their  work  affects  all  the  others 

with  which  they  interact. 

The  military,  with  its  orientation  to  outpatient  care,  has 
led  the  way  in  establishing  centralized  appointment  systems. 

Most  of  these  systems  have  been  automated  using  varying  sources 
of  computer  support.  A  study  conducted  by  R.  B.  Stuart  in  1972 
evaluated  the  central  appointment  systems  of  four  Army  hospitals. 
Stuart's  study  found  that  ninety-one  percent  of  the  patients 
believed  that  the  CAS  was  convenient  for  them,  while  sixty-two 
percent  of  the  staff  approved  of  the  CAS  operation. 26  Ninety- 
three  percent  of  the  patients  found  the  CAS  clerks  courteous, 
while  only  fifty-one  percent  thought  the  CAS  clerks  took  a 
personal  interest  in  them. 27  Nearly  three-quarters  of  the 
patients  thought  the  waiting  time  in  the  clinics  was 
reasonable . 28  Numerous  recommendations  were  made  by  Stuart  at 
the  completion  of  his  study.  His  overall  conclusion  was  that  in 
multi-specialty  clinics,  centralized  outpatient  appointment 
systems  are  generally  effective  and  efficient  mechanisms  for 
insuring  that  the  health  care  provider,  the  ambulatory  patient, 
and  his  medical  record  arrive  at  the  right  time  for  a 
patient/ provider  encounter  with  a  minimum  of  waiting  by  either 
29 


party . 
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Singer,  Rossfeld,  and  Van  Hall  conducted  a  study  of  a 
centralized  appointment  system  In  a  civilian  Institution,  Harbor 
General  Hospital,  In  197^.  Their  findings  revealed  that  the 
Introduction  of  a  centralized  appointment  system  resulted  In  a 
number  of  positive  changes  In  the  way  the  hospital  provided  care 
to  outpatients  by  making  a  modest  reduction  In  average  waiting 
time  and  no-show  rates. 

There  Is  no  evidence  that  any  studies  have  been  conducted  to 
compare  patients'  satisfaction  with  appointment  systems. 

Stuart's  study  briefly  looked  at  some  patient  satisfaction 
Issues,  but  not  In  very  much  depth.  Chaffee  addressed  the 
patient  satisfaction  Issue  In  a  study  conducted  In  1981  at  Dwight 
David  Elsenhower  Army  Medical  Center;  however,  no  data  was 
collected  to  compare  patients'  satisfaction  with  centralized 
versus  decentralized  systems. 31  There  Is  no  evidence  of  any 
research  to  determine  If  there  Is  a  difference  In  satisfaction 
level  of  patients  using  a  centralized  appointment  system  versus 
patients  using  a  decentralized  appointment  system.  A  large  Army 
Medical  Center  Is  the  Ideal  setting  to  conduct  this  research 
since  outpatient  appointments  are  made  using  both  a  centralized 
and  a  decentralized  appointment  system. 

Research  Methodology 

The  first  step  In  conducting  the  research  for  the  specific 
problem  was  familiarization  with  the  current  outpatient 
appointment  system  being  utilized  at  MAMC.  This  process  Included 
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interviewing  the  Patient  Appointment  System  supervisor  and 
appointment  clerks,  the  Administrative  Coordinator  for  the  Deputy 
Commander  for  Clinical  Services,  the  Automation  Management 
Officer,  and  the  Deputy  Commander  for  Administration  to  ascertain 
how  the  current  system  functions  and  what  problems  relating  to 
the  operation  of  the  system  exist. 

After  completion  of  the  preliminary  research  of  the 
appointment  system,  determination  of  an  appropriate  sample  size 
was  made,  utilizing  tables  taken  from  Backstrom  and  Hursh-Cesar 
and  methods  recommended  by  Lazerwitz . 32 , 33  Utilizing  a  ninety- 
five  percent  confidence  level  and  accepting  a  five  percent 
tolerated  error,  the  Backstrom  and  Hursh-Cesar  tables  required  a 
minimum  sample  size  of  384.  This  number  is  consistent  with  the 
Lazerwitz  methodology  in  which  the  sample  size  is  computed  as  the 
reciprocal  of  k  squared,  where  k  is  the  desired  precision  level 
about  .5  at  a  95}  confidence  level.  Using  this  method,  if  a  five 
percentage  point  maximum  confidence  interval  is  desired,  sample 
size  was  determined  to  be  400.  In  order  to  insure  that  400 
respondents  actually  complete  the  survey,  300  patients  were 
selected,  since  a  return  rate  greater  than  seventy  percent  was 
anticipated . 

Utilizing  this  sample  size  of  400  for  the  two  categories  of 
appointment  systems  (centralized  and  decentralized)  a 
determination  of  the  proportion  of  the  total  sample  survey  to  be 
completed  in  each  clinic  was  made.  This  process  was  accomplished 
utilizing  the  following  procedures: 
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1.  A  calculation  of  the  average  dally  clinic  visits  (ADCV)  for 
those  clinics  utilizing  an  appointment  system  (both  centralized 
and  decentralized)  was  made  utilizing  historical  workload  data. 
Outpatient  clinics  not  utilizing  an  appointment  system,  such  as 
Emergency  Room,  Acute  Illness  Clinic,  and  the  Troop  Medical 
Clinics  were  excluded  from  the  calculations. 

2.  The  ADCV  for  each  clinic  was  then  compared  with  the  total 
ADCV  (2016)  to  establish  a  percentage  of  the  total  for  each 
clinic . 

3.  The  percentage  for  each  clinic  was  then  applied  to  the  total 
sample  size  of  500  to  establish  the  number  of  surveys  to  be 
completed  in  each  clinic  on  the  day  of  the  survey.  This  number 
was  labelled  n^* 

4.  On  the  day  of  the  survey  the  number  of  patient  appointments 
for  that  day  in  each  clinic,  labelled  dc,  was  divided  by  n^  to 
give  a  value  of  k  (dQ/nQ=k).  Every  kth  patient  who  arrived  for 
an  appointment  on  the  day  of  the  survey  was  administered  a 
survey.  This  systematic  sampling  method  has  been  found  to  be 
equivalent  to  a  random  sample  in  cases  where  the  sample  frame  is 
not  cyclic,  and  was  much  easier  to  Implement  in  the  clinic 
setting  than  a  random  number  or  lottery-type  selection . 34 , 35  The 
values  for  each  clinic  is  presented  in  Table  1 . 


16 


TABLE  1 

IN-HOSPITAL  SURVEY  DISTRIBUTION  TABLE 


CLINIC 

AVERAGE 

DAILY 

VISITS 

%  OF 
TOTAL 
CLINIC 
VISITS 

#  OF 

SURVEYS 

REQUIRED 

SAMPLING 
VALUE  K 

FOR 

SURVEY  DIST 

Adolescent 

40 

2.0 

10 

4 

Allergy 

63 

3.0 

16 

4 

Cardiology 

74 

4.0 

18 

4 

Dermatology 

61 

3.0 

15 

4 

Endocrine 

36 

2.0 

9 

4 

Family  Practice 

187 

9.0 

46 

4 

Gastroenterology 

53 

3.0 

13 

4 

General  Surgery 

42 

2.0 

1 1 

4 

Gynecology 

149 

7.0 

37 

4 

Hematology 

19 

1.0 

5 

4 

Initial  Visit 

93 

5.0 

23 

4 

Internal  Medicine 

74 

4.0 

18 

4 

Nephrology 

24 

1.0 

6 

4 

Neurology 

35 

2.0 

9 

4 

Obstetrics 

127 

6.0 

31 

4 

Oncology 

83 

4.0 

21 

4 

Ophthalmology 

43 

2.0 

1 1 

4 

Optometry 

94 

5.0 

23 

4 

Orthpaedics 

98 

5.0 

24 

4 

Otolaryngology 

55 

3.0 

14 

4 
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Table  1  (Continued) 


CLINIC 

AVERAGE 

DAILY 

VISITS 

%  OF 
TOTAL 
CLINIC 
VISITS 

#  OF 

SURVEYS 

REQUIRED 

SAMPLING 

VALUE  K 

FOR 

SURVEY  DIST 

Pediatrics 

281 

14.0 

70 

4 

Plastic  Surgery 

26 

1.0 

6 

4 

Podiatry 

27 

1.0 

6 

4 

Pulmonary  Disease 

46 

2.0 

1 1 

4 

Rheumatology 

28 

1.0 

7 

4 

Social  Work 

19 

1 .0 

5 

4 

Speech/ Audiology 

23 

1.0 

6 

4 

Urology 

55 

3.0 

14 

4 

Well  Child 

61 

3.0 

15 

4 

TOTALS 

2016 

100 

500 

In  determining  the  required  number  of  mailings  for  the 
survey  of  retirees  the  Retired  Services  Officers  for  Fort  Lewis 
and  McChord  Air  Force  Base  were  contacted  and  asked  to  provide  a 
listing  of  registered  retirees  in  the  immediate  area  of  MAMC. 
Initial  queries  of  the  Fort  Lewis  Retired  Services  Officer 
revealed  that  there  were  approximately  14,000  Army  retirees  in 
the  MAMC  catchment  area.^^  The  Air  Force  Retired  Services 
Officer  estimated  a  total  of  5,000.  A  total  of  20,700  preprinted 
mailing  labels  were  received  from  both  the  Fort  Lewis  and  McChord 
Retired  Services  Offices.  Based  on  the  resources  available,  the 
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target  sample  size,  and  the  expected  return  rate  it  was 
determined  to  utilize  every  twelfth  label  which  generated  a  total 
of  1725  mail-outs. 

Prior  to  initiation  of  the  survey  process,  the  survey 
instrument  was  evaluated  for  validity.  The  validation  process 
included : 

1.  Review  of  the  survey  by  LTC  Patricia  Basta,  Chief,  Clinical 
Nursing  Services,  Department  of  Nursing,  MAMC.  LTC  Basta  has  a 
PhD  in  nursing  with  several  years  of  experience  in  research 
design  and  is  the  expert  for  Department  of  Nursing  personnel 
conducting  research  at  MAMC.  After  reviewing  the  survey 
Instruments  her  only  comment  was  that  the  survey  must  be 
adequately  pre-tested  to  insure  question  clarity. 

2.  Review  of  the  survey  instrument  by  a  panel  of  individuals  who 
had  previous  experience  working  with  the  PAS  at  MAMC.  This  panel 
recommended  several  changes  to  Include  (1)  reducing  the  length  of 
the  questionnaire  and  addition  of  emphasis  markings  for  clarity 
of  directions,  (2)  changing  all  references  to  the  "Central 
Appointment  System"  to  read  "Patient  Appointment  System,"  and  (3) 
clarifying  the  question  about  waiting  time  to  make  an  appoint¬ 
ment.  It  was  also  recommended  that  coordination  for  the  actual 
survey  in  the  clinics  be  made  with  each  department's 
administrative  assistant  for  better  control. 

3.  Administration  of  a  pretest  to  the  patient  population  to 
determine  if  the  questions  being  asked  were  understandable  and 
could  be  answered  easily.  The  in-hospital  survey  pretest  was 
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completed  using  a  random  sample  of  patients  In  the  OB/GYN  and 
Pediatric  Clinics.  The  ARC  Volunteer  administered  the 
questionnaire  to  individuals  and  then  interviewed  them  at  the 
completion  of  the  questionnaire.  A  similar  process  was  completed 
for  the  mall-out  survey.  This  questionnaire  was  administered  to 
a  group  of  retirees  who  attended  a  retiree  council  meeting.  The 
ARC  Volunteer  completed  this  process  to  include  the  interview  of 
each  respondent  after  completion  of  the  questionnaire. 

4.  Evaluation  of  information  received  from  the  pretest  to 
determine  if  the  survey  questions  were  clear  and  understandable. 
In  addition,  review  of  the  open-ended  questions  were  conducted  to 
evaluate  the  possibility  of  including  additional  questions  not 
considered  in  the  initial  survey.  Subjective  information 
provided  by  the  individual  administering  the  pretest  was  also 
considered  in  evaluation  of  the  structure  of  the  survey 
instrument . 


5  . 
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letter,  using  a  different  type  face  to  make  it  easier  to  read; 

(3)  redesign  of  question  three  directing  the  respondent  to  a 
single  page  instead  of  three  different  pages;  (4)  combining  the 
originally  designed  question  for  decentralized  appointments  and 
centralized  appointments  into  a  single  sheet;  and  (5)  elimination 
of  page  four  in  the  original  questionnaire.  This  question  was 
combined  with  question  three  on  the  first  page.  The  overall 
effect  of  the  changes  was  to  reduce  the  original  questionnaire 
from  six  pages  to  four  pages  and  insert  more  directions  and 
emphasis  markings  for  clarity  of  instructions.  The  redesign  of 
the  mail-out  survey,  based  on  pretest  input,  was  basically  the 
same  as  the  in-hospital  survey.  The  other  change  made  to  the 
mail-out  survey  was  to  redesign  the  questions  for  individuals  not 
using  the  outpatient  services.  Pretest  input  indicated  that  the 
directions  on  the  original  questionnaire  were  potentially 
confusing.  Some  of  the  questions  were  reworded  and  numerous 
directions  and  emphasis  markings  were  placed  on  the  revised 
questionnaire.  Again,  the  original  survey  of  seven  pages  was 
reduced  to  five  pages  and  a  different  type  face  was  utilized  to 
allow  for  ease  of  reading  by  the  respondent. 

Upon  completion  of  the  survey  instrument  validation  the 
research  proposal  was  presented  to  the  MAMC  Institution  Review 
Board  for  approval  to  conduct  human  subject  research.  Approval 
was  granted  in  May,  1985. 

Administration  of  the  In-hospltal  survey  was  completed 
utilizing  a  Red  Cross  volunteer  to  coordinate  completion  of 
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requirements.  The  mechanism  for  administering  the  survey 
Included : 

1.  Surveys  were  distributed  In  each  outpatient  clinic  having  an 
appointment  system.  Distribution  of  the  Individual  surveys  to 
patients  was  determined  by  the  clinic  receptionist  based  on  the 
number  of  completed  surveys  required  for  the  sample  size  and  the 
value  of  k  used  for  systematic  random  sampling.  All  of  this 
Information  was  provided  In  an  Instruction  sheet  given  to  the 
receptionist . 

2.  The  completed  patient  surveys  were  sealed  In  an  envelope  and 
returned  to  the  receptionist  by  the  respondent.  The  ARC  volunteer 
was  available  to  assist  the  clinic  receptionist  in  this  process. 

3.  The  completed  surveys  were  picked  up  by  the  volunteer  and 
returned  to  the  author  for  analysis  of  responses. 

The  preferred  method  of  conducting  the  survey  would  have 
been  to  accomplish  the  task  of  completing  questionnaires  in  each 
respective  area  on  one  chosen  day.  Due  to  the  large  number  of 
clinics  and  the  physical  layout  of  MAMC,  completing  the  survey  in 
one  day  was  not  feasible.  Responses  may  have  been  biased  by  the 
day  of  survey  administration  due  to  the  different  influences  such 
as  weather  conditions  or  day  of  the  week.  This  relatively  minor 
potential  limitation  was  accepted  In  order  to  provide  closer 
supervision  of  survey  administration  in  each  clinic.  The  entire 
in-hospltal  survey  took  five  working  days  to  complete. 

The  mall-out  survey  was  completed  using  ARC  volunteers. 

They  Individually  placed  a  survey  questionnaire  and  a  postage 
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paid,  pre-addressed ,  return  envelope  In  each  mailing  envelope  and 
selected  the  appropriate  address  label  provided  by  the  Retired 
Services  offices.  Upon  receipt  of  each  completed  questionnaire 
the  envelopes  were  opened  and  the  surveys  were  reviewed  for 
completeness . 

Analysis  of  the  data  was  accomplished  by  compiling  the 
responses  to  individual  questions.  A  numerical  value  from  one  to 
five  was  assigned  to  each  response  on  the  satisfaction  scales. 

In  addition,  an  overall  satisfaction  score  was  computed  for  each 
respondent  by  averaging  responses  for  all  six  questions.  Chi- 
Square  and  T-Tests  were  performed  to  determine  if  there  is  a 
significant  difference  In  the  level  of  satisfaction  between 
patients  using  a  centralized  versus  a  decentralized  appointment 
system.  A  .05  level  of  significance  was  utilized.  The  direction 
of  results  (favoring  either  type  of  system)  was  not  hypothesized. 
In  addition,  analysis  of  each  individual  question  relating  to  the 
appointment  system  was  conducted  to  determine  the  practical 
significance  of  each  specific  issue  such  as  waiting  time  for  an 
appointment,  lag  time  from  making  the  appointment  to  actual 
appointment  date,  the  way  the  patient  was  treated  by  the 
appointment  clerk,  information  given  by  the  appointment  clerk, 
and  the  overall  opinion  of  the  appointment  system. 
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II.  DISCUSSION 


Survey  Return  Rate 

A  very  important  consideration  in  any  survey  research  is  the 
return  rate.  In  this  study  the  return  rate  was  extremely  large 
in  comparison  to  the  expected  rates  and  most  of  the  rates 
published  in  the  literature  on  survey  research . ^ » 2 » 3 f ^  As  stated 
in  the  criteria  section,  a  return  rate  of  seventy  percent  for  the 
in-hospital  survey  and  twenty  percent  for  the  mail-out  survey 
would  be  required  to  consider  the  survey  process  as  valid.  A 
total  of  452  of  the  500  survey  questionnaires  for  the  in-hospital 
survey  were  returned  for  a  rate  of  90.4  percent.  Pre-survey 
coordination  contributed  to  the  large  in-hospital  survey  return 
rate.  Prior  to  the  actual  date  for  the  in-hospital  survey, 
personal  contact  was  made  with  each  clinic  receptionist  in  those 
clinics  designated  to  participate.  A  complete  explanation  of  the 
research  project  and  the  survey  instrument  was  presented.  A 
second  factor  influencing  the  large  return  rate  was  that  the 
actual  survey  was  conducted  over  a  five  day  period  and  therefore 
allowed  the  researcher  and  his  assistant  to  be  physically  present 
during  a  portion  of  the  time  when  the  surveys  were  being 
completed  in  each  clinic.  This  allowed  the  receptionists  to  ask 
any  questions  about  passing  out  the  surveys  and  Insured  that  the 
surveys  were  distributed  properly  and  returned  by  the 
respondents.  Each  day  at  the  completion  of  the  survey,  the 
researcher  picked  up  the  completed  surveys.  In  those  clinics 
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where  all  of  the  required  surveys  were  not  returned,  the 


receptionist  was  asked  the  reason  for  surveys  non-completion. 

The  majority  of  reasons  related  to  the  issue  of  insufficient  time 
to  dedicate  to  distribution  of  surveys.  In  no  instances  were  the 
surveys  not  completed  because  of  patient  refusal.  Had  the 
receptionists  distributed  all  of  their  required  surveys  an  even 
greater  return  rate  would  have  been  anticipated.  Table  2 
presents  the  number  of  surveys  completed  in  each  individual 
clinic . 


TABLE  2 

FREQUENCY  OF  SURVEYS  COMPLETED  IN  EACH  CLINIC 


CLINIC 

NUMBER  OF 
SURVEYS 
COMPLETED 

PERCENTAGE  OF 
TOTAL 

COMPLETED  SURVEYS 

ADOLESCENT 

10 

2.2 

ALLERGY 

1  1 

2.4 

CARDIOLOGY 

16 

3.5 

DERMATOLOGY 

14 

3.1 

ENDOCRINE 

9 

2.0 

FAMILY  PRACTICE 

40 

8.8 

GASTROENTEROLOGY 

12 

2.7 

GENERAL  SURGERY 

1  1 

• 

CNJ 

GYNECOLOGY 

20 

4.4 

HEMATOLOGY 

5 

1  .  1 

INITIAL  VISIT 

23 

5.1 

INTERNAL  MEDICINE 

17 

U) 

• 

00 

NEPHROLOGY 

6 

1.3 
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Table  2 

( Continued ) 

CLINIC 

NUMBER  OF 
SURVEYS 
COMPLETED 

PERCENTAGE  OF 
TOTAL 

COMPLETED  SURVEYS 

NEUROLOGY 

8 

1.8 

OBSTETRICS 

27 

6.0 

ONCOLOGY 

20 

4.4 

OPTHALMOLOGY 

1 1 

2.4 

OPTOMETRY 

22 

4.9 

ORTHOPEDICS 

24 

5.3 

OTOLARYNGOLOGY 

14 

3.1 

PEDIATRICS 

64 

14.2 

PLASTIC  SURGERY 

6 

1.3 

PODIATRY 

6 

1.3 

PULMONARY  DISEASE 

1  1 

2.4 

RHEUMATOLOGY 

7 

1.5 

SOCIAL  WORK 

4 

0.9 

SPEECH  AUDIOLOGY 

5 

1  . 1 

UROLOGY 

14 

3.1 

WELL  CHILD 

15 

3.3 

TOTAL 

452 

100.0 

Comparing  the  frequencies  listed  In  Table  2  with  those  of 
Table  1  Indicates  that  six  clinics  failed  to  complete  only  one 
survey  each.  Of  the  remaining  clinics  the  largest  Incompletlon 


occurred  In  the  Gynecology  Clinic  where  seventeen  surveys  were 
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not  completed.  When  the  receptionist  of  this  clinic  was  asked 
why  surveys  were  not  completed,  she  stated  that  during  the  day  of 
the  survey  she  became  very  busy  and  had  several  of  the  clinic 
staff  helping  her.  She  failed  to  instruct  these  individuals 
about  passing  out  the  survey.  A  similar  situation  occurred  in 
the  Family  Practice  Clinic  where  six  surveys  were  returned 
without  being  filled  out. 

The  mail~out  survey  consisted  of  1725  questionnaires.  Of 
this  total,  eighty-two  of  the  mailings  were  returned  to  the 
surveyor  with  an  insufficient  or  incorrect  address.  Assuming 
that  the  remaining  1643  surveys  were  received  by  the  correct 
individuals,  the  calculated  return  rate  for  the  867  surveys  that 
were  received  is  52.7  percent.  It  appears  that  the  large  return 
rate  is  indicative  of  the  respondents'  eagerness  to  provide 
information  on  an  issue  which  is  very  timely  and  controversial. 

In  both  surveys  the  fact  that  the  survey  introductory  letter  was 
signed  by  the  Commander  had  a  positive  effect  on  the  return  rate. 
This  is  exhibited  by  the  fact  that  a  number  of  surveys  were 
returned  with  personal  notes  and  letters  addressed  to  Brigadier 
General  Powell.  A  third  reason  for  the  high  return  is  the  fact 
that  the  pretest  process  provided  the  researcher  with  a  survey 
Instrument  which  was  easy  to  understand  and  very  simple  to 
follow.  The  use  of  the  Retired  Services  Offices'  reasonably  up- 
to-date  mailing  lists  (to  Include  preprinted  labels)  also 
contributed  to  the  increased  return  rate.  Only  four  percent  of 
the  addresses  provided  were  found  to  be  Incorrect. 
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Description  of  the  Sample  Population 
The  sample  population  for  the  In-hospltal  survey  was 
distributed  across  the  four  main  categories  of  patient  status 
with  a  few  responses  in  the  Family  Member  of  Deceased  Service 
Member  and  Other  categories.  There  were  a  total  of  four 
respondents  who  did  not  answer  the  question  on  status.  The 
frequency  of  status  of  respondents  is  presented  in  Table  3. 

TABLE  3 


ELIGIBILITY  STATUS  OF  RESPONDENTS  FOR  IN-HOSPITAL  SURVEY 


ELIGIBILITY  STATUS 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

CUM 

FREQ 

(PCT) 

ACTIVE  DUTY  SERVICE  MEMBER 

84 

18.6 

18.8 

18.8 

FAMILY  MEMBER  ACTIVE  DUTY 
SERVICE  MEMBER 

147 

32.5 

32.8 

51  .6 

RETIRED  SERVICE  MEMBER 

96 

21 .2 

21  .4 

73.0 

FAMILY  MEMBER  RETIRED 
SERVICE  MEMBER 

107 

23.7 

23.9 

96.9 

FAMILY  MEMBER  DECEASED 
SERVICE  MEMBER 

12 

2.7 

2.7 

99.6 

OTHER 

2 

0.4 

0.4 

100.0 

NO  RESPONSE 

4 

0.9 

N/A 

N/A 

TOTAL 

452 

100.0 

100.0 

In  evaluating  the  amount  of  time  the  respondents  had  used 
the  outpatient  clinic  services,  53.9  percent  of  respondents 
answered  that  they  had  used  these  services  for  a  period  of  less 
than  three  years,  and  46.1  percent  stated  that  they  had  used  the 
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clinic  services  for  a  period  greater  than  three  years.  A  total 
of  four  individuals  did  not  respond  to  this  question.  Table  4 
lists  response  frequencies  for  the  amount  of  time  respondents  had 
used  the  outpatient  services  at  MAMC. 


TABLE  4 

AMOUNT  OF  TIME  IN-HOSPITAL  SURVEY  RESPONDENTS 
HAD  USED  OUTPATIENT  SERVICES 


TIME  USED  OUTPATIENT 
SERVICES 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

CUM 

FREQ 

(PCT) 

LESS  THAN  6  MONTHS 

72 

15.0 

16.1 

16.1 

6-11  MONTHS 

63 

13.9 

14.1 

30.2 

1-3  YEARS 

106 

23.5 

23.7 

53.9 

GREATER  THAN  3  YEARS 

207 

45.8 

46.1 

100.0 

NO  RESPONSE 

4 

0.9 

N/A 

N/A 

TOTAL  452 

100.0 

100.0 

The  method  used  to  make  the  appointment  is  the  most 
important  information  received  from  the  respondents.  In  order  to 
be  able  to  compare  satisfaction  with  different  appointment 
systems,  it  was  necessary  to  know  to  which  system  respondents 
were  referring.  Response  frequencies  related  to  the  method  of 


making  the  most  recent  appointment  are  presented  in  Table  5. 


TABLE  5 


METHOD  USED  TO  MAKE  APPOINTMENT  FOR  IN-HOSPITAL  SURVEY 


METHOD  OF  MAKING 

APPOINTMENT 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

CUM 

FREQ 

(PCT) 

CALLED  PAS 

125 

27.7 

27.9 

27.9 

CALLED  CLINIC 

146 

32.3 

32.6 

60.5 

CLINIC  IN  PERSON 

91 

20 . 1 

20.3 

80.8 

WALK-IN 

10 

2.2 

2.2 

83.0 

POSTCARD  MAIL-IN 

22 

4.9 

4.9 

94.2 

PHYSICIAN  MADE  APPOINTMENT 

28 

6.2 

6.3 

94.9 

EMERGENCY  ROOM  VISIT 

3 

0.7 

0.7 

100.0 

OTHER  METHOD 

23 

5.1 

5.1 

100.0 

NO  RESPONSE 

4 

0.9 

N/A 

N/A 

TOTAL 

452 

100.0 

100.0 

Approximately  twenty-eight  percent  had  made  their  most 
recent  appointment  by  calling  the  PAS,  while  52.9  percent  had 
made  their  appointment  through  the  clinic  either  by  calling  or 
visiting  the  clinic  In  person.  Various  other  methods  were  given 
by  the  respondents,  to  Include  4.9  percent  who  made  the 
appointment  by  using  a  mall-ln  postcard,  6.3  percent  who  had 
their  appointment  made  by  their  physician,  and  2.2  percent  who 
were  seen  on  a  walk-ln  basis.  A  total  of  four  Individuals  did 
not  respond  to  this  question. 

The  sample  population  for  the  mall-out  survey  was 
distributed  across  three  main  categories  of  patient  status  not 
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including  the  two  responses  in  the  Other  category.  Table  6  shows 
eligibility  status  frequency  for  the  mail-out  survey. 


TABLE  6 

STATUS  OF  RESPONDENTS  FOR  MAIL-OUT  SURVEY 


ELIGIBILITY  STATUS 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

CUM 

FREQ 

(PCT) 

RETIRED  SERVICE  MEMBER 

649 

74.9 

76.2 

76.2 

FAMILY  MEMBER  RETIRED 
SERVICE  MEMBER 

113 

13.0 

13.3 

89.5 

FAMILY  MEMBER  DECEASED 
SERVICE  MEMBER 

87 

10.0 

10.2 

99.7 

OTHER 

2 

.2 

.3 

100.0 

NO  RESPONSE 

16 

1.9 

N/A 

N/A 

TOTAL 

867 

100.0 

100.0 

Approximately  seventy-six  percent  of  the  respondents  were 
retired  service  members,  while  23.4  percent  were  family  members 
of  retired  or  deceased  service  members.  The  large  proportion  of 
responses  of  retired  service  members  is  due  to  the  fact  that  all 
surveys  were  addressed  to  the  retired  service  member  and  not  to 
individual  family  members.  A  total  of  sixteen  individuals  did 
not  respond  to  this  question. 

Responses  to  the  question  concerning  the  amount  of  time 
respondents  had  used  the  outpatient  clinic  services  at  MAMC  were 


distributed  among  five  categories,  Including  a  response  for  those 
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who  had  never  used  the  system.  Mail-out  survey  response 
frequencies  for  this  question  are  presented  in  Table  7. 


TABLE  7 

AMOUNT  OF  TIME  MAIL-OOT  SURVEY  RESPONDENTS 
HAD  USED  THE  OUTPATIENT  SERVICES 


TIME  USED  OUTPATIENT 
SERVICES 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

CUM 

FREQ 

(PCT) 

LESS  THAN  6  MONTHS 

25 

2.9 

3.0 

3.0 

6-11  MONTHS 

23 

2.7 

2.8 

5.8 

1-3  YEARS 

48 

5.5 

5.7 

11  .5 

GREATER  THAN  3  YEARS 

574 

66.2 

68.6 

80.1 

NEVER  USED  SYSTEM 

166 

19.1 

19.9 

100.0 

NO  RESPONSE 

31 

3.6 

N/A 

TOTAL 

867 

100.0 

100.0 

Approximately  twelve  percent  had  used  the  system  for  less 
than  three  years,  while  68.6  percent  had  used  the  system  more 
than  three  years,  and  19.9  percent  bad  never  used  the  system. 
Thirty-one  individuals  did  not  respond  to  this  question.  The  166 
individuals  who  had  never  used  the  system  gave  five  different 
reasons  for  non-use  of  the  outpatient  services.  Table  8  presents 
the  number  of  non-users  and  the  reasons  for  not  using  the 


outpatient  services  at  MAMC. 
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TABLE  8 

REASONS  FOR  NOT  USING  THE  OUTPATIENT  SERVICES  AT  MAMC 


REASON 

NUMBER 

PERCENTAGE 

DISTANCE 

88 

53.0 

DISSATISFIED  WITH  APPOINTMENT  SYSTEM 

10 

6.0 

OTHER  MEDICAL  INSURANCE 

23 

13.9 

NEVER  BEEN  SICK 

25 

15 . 1 

OTHER 

21 

12.0 

TOTAL 

167 

100.0 

Over  half  (fifty-three  percent)  of  these  respondents  had  not 
used  the  outpatient  services  because  MAMC  was  located  too  far 
from  their  home.  Only  six  percent  (ten  people)  stated  that  their 
non-use  was  due  to  dissatisfaction  with  the  appointment  system. 
Approximately  fourteen  percent  responded  that  they  had  medical 
Insurance  coverage  for  outpatient  services  in  the  civilian  health 
care  community.  It  is  interesting  to  note  that  15.1  percent 
responded  that  they  had  never  been  sick,  therefore  not  requiring 
the  outpatient  services  available  to  them. 

In  response  to  the  question  concerning  the  method  used  to 
make  their  last  appointment,  41.9  percent  of  the  mail-out  survey 
respondents  replied  that  they  had  called  the  PAS,  while  thirty 
percent  made  their  appointment  by  calling  or  visiting  the  clinic 
in  person.  Approximately  three  percent  utilized  the  mail-in  card 
system  and  approximately  one  percent  responded  to  each  of  the 
three  categories  which  Included  walk-in,  physician,  and  emergency 
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room.  The  majority  of  "not  applicable"  responses  were  from  those 
individuals  who  had  responded  that  they  had  never  used  the 
outpatient  clinic  services  at  MAMC.  Individuals  not  responding 
to  this  question  made  up  2.7  percent  of  the  sample  population. 
Table  9  shows  the  frequencies  of  responses  to  the  question 
concerning  method  of  making  the  appointment. 


TABLE  9 

METHOD  USED  TO  MAKE  APPOINTMENT  FOR  MAIL-OUT  SURVEY 


METHOD  OF  MAKING 
APPOINTMENT 

ABSOLUTE 

FREQ 

RELATIVE 

FREQ 

(PCT) 

ADJUSTED 

FREQ 

(PCT) 

IQ  M 
IQ  ^ 
■H»H 

CALLED  PAS 

354 

40.8 

41.9 

41  .9 

CALLED  CLINIC 

133 

15.3 

15.8 

57.7 

CLINIC  IN-PERSON 

120 

13.8 

14.2 

71.9 

WALK-IN 

10 

1.2 

1.2 

73.1 

POSTCARD  MAIL-IN 

24 

2.8 

2.9 

76.0 

PHYSICIAN  MADE  APPOINTMENT 

10 

1.2 

1.2 

77.2 

EMERGENCY  ROOM  VISIT 

9 

1.0 

1  .0 

78.2 

OTHER  METHOD 

13 

1.5 

1.5 

79.7 

NOT  APPLICABLE 

170 

19.6 

20.3 

100.0 

NO  RESPONSE 

24 

2.7 

N/A 

N/A 

TOTAL 

867 

100.0 

100.0 

General 

Satisfaction 

Levels 

Analysis  of  the  responses  to  the  seven  questions  concerning 
satisfaction  with  specific  areas  of  the  appointment  system 
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provides  some  insight  Into  the  satisfaction  levels  of  the  In- 
hospital  and  mail-out  survey  groups.  The  specific  areas 
addressed  Include:  (1)  the  amount  of  time  it  takes  to  make  an 
appointment,  (2)  the  way  the  respondent  was  treated  by  the 
appointment  clerk,  (3)  the  time  and  date  of  the  appointment,  (4) 
the  amount  of  time  between  making  the  appointment  and  the  actual 
appointment  date,  (5)  the  Information  provided  by  the  appointment 
clerk,  (6)  the  respondents'  overall  opinion  of  the  appointment 
system,  and  (7)  comments  on  how  to  change  the  existing 
appointment  system.  Each  of  these  areas  will  be  analyzed 
comparing  responses  of  the  in-hospital  and  mail-out  groups. 
Overall  satisfaction  percentages  are  calculated  by  combining  the 
"satisfied"  and  "very  satisfied"  responses.  Combining  the  "very 
dissatisfied"  and  "dissatisfied"  response  percentages  gives  an 
overall  dissatisfaction  indicator. 

In-hospital  and  mall-out  group  responses  to  the  question 
concerning  the  amount  of  time  it  takes  to  make  an  appointment 
differed.  Table  10  presents  the  response  frequencies  for  the 
time  to  make  the  appointment  question. 
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TABLE  10 

LEVEL  OF  SATISFACTION  WITH  THE  AMOUNT  OF  TIME 


IT  TAKES 

TO  MAKE  AN  APPOINTMENT 

ADJUSTED 

ABSOLUTE 

FREQ 

LEVEL  OF  SATISFACTION 

FREQ 

(PCT) 

VERY  DISSATISFIED 


In-Hospital 

19 

4.8 

4.8 

Mail-Out 

77 

12.1 

12.1 

DISSATISFIED 

In  Hospital 

29 

7.3 

12.1 

Mail-Out 

100 

15.7 

27.8 

NEITHER 

In-Hospital 

28 

7.0 

19.1 

Mail-Out 

60 

9.4 

37.2 

SATISFIED 

In-Hospital 

135 

33.9 

53.0 

Mall-Out 

201 

31.6 

68.8 

VERY  SATISFIED 

In-Hospital 

187 

47.0 

100.0 

Mail-Out 

OVERALL  SATISFACTION 

In-Hospital  -- 
Mail-Out 

PERCENTAGE 

80.9 

62.8 

199 

31.2 

100.0 

Almost  twenty-eight  percent  of  the  mail-out  group  were 
dissatisfied.  In  comparison,  only  12.1  percent  of  the  in- 
hospital  group  were  dissatisfied.  Approximately  one-third  of 
both  groups'  responses  were  "satisfied"  but  the  in-hospital 
groups'  response  of  "very  satisfied"  exceeded  that  of  the 
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mail-out  group  by  15.8  percent.  Overall,  80.9  percent  of  the  in- 
hospital  group  and  62.8  percent  of  the  mail-out  group  were 
satisfied. 

Approximately  five  percent  of  the  mail-out  group  and  two 
percent  of  the  in-hospital  group  were  dissatisfied  with  the 
treatment  provided  by  the  appointment  clerk.  Although  both 
groups  were  highly  satisfied,  tne  in-hospital  group  again  showed 
higher  satisfaction  levels.  The  overall  satisfaction  percentages 
for  the  in-hospital  and  mail-out  group  were  93.9  and  88.0  percent 
respectively.  Response  frequencies  for  the  question  concerning 
treatment  by  the  appointment  clerk  are  presented  at  Table  11. 


TABLE  11 

LEVEL  OF  SATISFACTION  WITH  TREATMENT  BY  APPOINTMENT  CLERK 


LEVEL  OF  SATISFACTION 

ABSOLUTE 

FREQ 

ADJUSTED 

FREQ 

(PCT) 

IQ  ^ 
uZ  31 

VERY  DISSATISFIED 

In-Hospital 

1 

0.3 

0.3 

Mail-Out 

9 

1.4 

1.4 

DISSATISFIED 

In-Hospital 

7 

1.8 

2. 1 

Mail-Out 

20 

3.1 

4.5 

NEITHER 

In-Hospital 

16 

4.0 

6.1 

Mail-Out 

M7 

7.4 

11.9 

SATISFIED 

In-Hospital 

97 

24.4 

30.5 

Mail-Out 

229 

36.0 

47.9 
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Table  11  (Continued) 


ADJUSTED  CUM 
ABSOLUTE  FREQ  FREQ 

LEVEL  OF  SATISFACTION _ FREQ _ (PCT)  (PCT) 

VERY  SATISFIED 

In-Hospital  276  69.5  100.0 

Mail-Out  331  52.1  100.0 

OVERALL  SATISFACTION  PERCENTAGE 

In-Hospital  93.9 
Mail-Out  88.0 


The  question  concerning  satisfaction  with  the  time  and  date 
of  the  appointment  showed  a  considerable  difference  of  opinion 
between  the  two  groups.  A  comparison  of  response  frequencies  is 
presented  in  Table  12. 

TABLE  12 

LEVEL  OF  SATISFACTION  WITH  THE 


ACTUAL  TIME  AND 

DATE  OF  APPOINTMENT 

LEVEL  OF  SATISFACTION 

ADJUSTED 

ABSOLUTE  FREQ 

FREQ  (PCT) 

CUM 

FREQ 

(PCT) 

VERY  DISSATISFIED 


In-Hospltal 

8 

2.0 

2.0 

Mail-Out 

54 

8.5 

8.5 

DISSATISFIED 

In-Hospltal 

28 

7.1 

9.1 

Mail-Out 

89 

13.9 

22.4 

NEITHER 

In-Hospital 

25 

6.3 

15.4 

Mail-Out 

69 

10.8 

33.2 
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Table  12 

( Continued ) 

ABSOLUTE 

ADJUSTED 

FREQ 

CUM 

FREQ 

LEVEL  OF  SATISFACTION 

FREQ 

(PCT) 

(PCT) 

SATISFIED 

In-Hospital 

133 

33.5 

48.0 

Mail-Out 

221 

34.6 

67.8 

VERY  SATISFIED 

In-Hospital 

203 

51.1 

100.0 

Mail-Out 

OVERALL  SATISFACTION 

In-Hospital 

Mail-Out 

PERCENTAGE 

84.6 

66.8 

206 

32.2 

100.0 

Approximately  twenty-two  percent  of  the  mail-out  group  were 
dissatisfied  with  this  aspect  of  the  system  compared  with  only 
9.1  percent  of  the  in-hospital  respondents.  Comparing  the 
"satisfied"  responses  of  the  two  groups  indicates  little 
difference,  while  almost  nineteen  percent  more  of  the  in-hospital 
group  gave  a  "very  satisfied"  response.  Overall,  84.6  percent  of 
the  in-hospital  group  were  satisfied  with  this  area,  while  66.8 
percent  of  the  mall-out  group  expressed  satisfaction. 

The  In-hospltal  group  was  more  satisfied  with  the  amount  of 
time  between  making  the  appointment  and  the  actual  appointment 
date.  Nearly  one  quarter  of  the  mail-out  group  expressed 
dissatisfaction  compared  to  7.8  percent  of  the  In-hospltal  group. 
Again  the  "satisfied"  responses  of  the  two  groups  consisted  of 
approximately  one-third  of  the  total  responses.  The  major 
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difference  between  the  two  groups  was  in  the  "very  satisfied" 
category  with  the  in-hospital  group  exceeding  the  mail-out  group 
by  21.4  percent.  This  difference  contributed  to  the  variation 
between  the  in-hospital  groups'  84.2  percent  overall  satisfaction 
and  the  mail-out  groups'  63.3  percent  overall  satisfaction. 

Table  13  presents  the  response  frequencies  for  the  amount  of  time 
between  making  the  appointment  and  the  actual  appointment  date 
question . 


TABLE  13 

LEVEL  OF  SATISFACTION  WITH  THE  AMOUNT  OF  TIME 
BETWEEN  MAKING  THE  APPOINTMENT 
AND  THE  ACTUAL  DATE  OF  THE  APPOINTMENT 


ADJUSTED  CUM 
ABSOLUTE  FREQ  FREQ 

LEVEL  OF  SATISFACTION  _ FREQ _ (PCT)  (PCT) 


VERY  DISSATISFIED 

In-hospital 

Mail-Out 

DISSATISFIED 

In-Hospital 

Mail-Out 

NEITHER 

In-Hospital 

Mail-Out 


10  2.5  2.5 

61  9.7  9.7 


2 1 V  5.3  7.8 

92'  14.6  24.3 


32  8.1  15.9 

79  12.5  36.8 


SATISFIED 


In-Hospital 

Mail-Out 


140  35.3  51.2 

226  35.8  72.6 


VERY  SATISFIED 


In-Hospital 

Mail-Out 


194 

174 


48.8 

27.4 


100.0 

100.0 
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Table  13  (Continued) 

OVERALL  SATISFACTION  PERCENTAGE 

In-Hospital  84.2 

Mail-Out  63.3 


Both  groups  were  generally  satisfied  with  the  information 
provided  by  the  appointment  clerk.  A  comparison  of  the  response 
frequencies  for  the  question  concerning  information  provided  by 
the  appointment  clerk  is  presented  in  Table  14. 


TABLE  14 

LEVEL  OF  SATISFACTION  WITH  THE  INFORMATION 
PROVIDED  BY  THE  APPOINTMENT  CLERK 


LEVEL  OF  SATISFACTION 

ABSOLUTE 

FREQ 

ADJUSTED 

FREQ 

(PCT) 

IQ  ^ 

VERY  DISSATISFIED 

In-Hospital 

4 

1 .0 

1 .0 

Mail-Out 

15 

2.4 

2.4 

DISSATISFIED 

In-Hospital 

5 

1.3 

2.3 

Mail-Out 

23 

3.6 

6.0 

NEITHER 

In-Hospital 

28 

7 . 1 

9.4 

Mail-Out 

50 

7.9 

13.9 

SATISFIED 

In-Hospital 

126 

32.1 

41.5 

Mall-Out 

275 

43.5 

57.4 

VERY  SATISFIED 


In-Hospital 

Mail-Out 


230 

269 


58.5 

42.6 


100.0 

100.0 
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Table  14  (Continued) 

OVERALL  SATISFACTION  PERCENTAGE 

In-Hospital  90.6 

Mail-Out  86.1 


Only  six  percent  of  the  mail-out  group  and  2.3  percent  of 
the  in-hospital  group  expressed  dissatisfaction  with  the 
information  provided  by  the  appointment  clerk.  The  overall 
satisfaction  percentages  were  90.6  for  the  in-hospital  group  and 
86.1  for  the  mail-out  group. 

The  pattern  of  greater  satisfaction  for  the  in-hospital 
group  is  repeated  for  overall  opinion  of  the  appointment  system. 
Approximately  one-fourth  (24.7  percent)  of  the  mail-out  group 
expressed  dissatisfaction  when  asked  to  give  their  overall 
opinion  of  the  appointment  system.  The  in-hospital  group 
responses  of  "very  satisfied"  exceeded  the  mall-out  group  in  the 
same  category  by  16.2  percent.  Overall  satisfaction  was  reported 
by  79.5  percent  of  the  in-hospital  group  and  61.9  percent  of  the 
mail-out  group.  In  both  groups  these  overall  satisfaction 
percentages  were  lower  than  the  satisfaction  percentages  for  each 
of  the  other  five  questions.  Table  15  provides  the  response 
frequencies  for  the  overall  opinion  question. 
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TABLE  15 


OVERALL 

OPINION  OF  THE  APPOINTMENT 
USED  BY  THE  RESPONDENT 

SYSTEM 

ADJUSTED 

CUM 

ABSOLUTE 

FREQ 

FREQ 

OVERALL  OPINION 

FREQ 

(PCT) 

(PCT) 

VERY  DISSATISFIED 


In-Hospital 

1  1 

2.8 

2.8 

Mail-Out 

65 

10.3 

10.3 

DISSATISFIED 

In-Hospital 

30 

7.6 

10.4 

Mail-Out 

91 

14.4 

24.7 

NEITHER 

In-Hospital 

40 

10.1 

20.5 

Mail-Out 

84 

13.3 

38.0 

SATISFIED 

In-Hospital 

135 

34.1 

54.6 

Mail-Out 

206 

32.7 

70.7 

VERY  SATISFIED 

In-Hospital 

180 

25.4 

100.0 

Mail-Out 

OVERALL  SATISFACTION 

In-Hospital 

Mail-Out 

PERCENTAGE 

79.5 

61.9 

184 

29.2 

100.0 

When  asked  to  provide  ways  to  improve  the  existing 
appointment  system,  38.8  percent  of  the  mail-out  group  and  16,9 
percent  of  the  in-hospital  group  responded  that  no  changes  should 
be  made.  Of  those  individuals  who  recommended  changes  to  the 


system,  the  most  frequent  response  of  both  groups  was  to  "reduce 


46 

the  amount  of  time  it  takes  to  make  an  appointment.”  The  next 
most  frequent  response  for  both  groups  was  "to  increase  the 
number  of  phone  lines  for  making  appointments."  Approximately 
four  percent  of  the  in-hospital  group  and  eight  percent  of  the 
mail-out  group  recommended  a  complete  centralization  of  the 
appointment  system,  while  no  in-hospital  respondents  and  only  0.6 
percent  of  the  mail-out  group  recommended  a  total  decentraliza¬ 
tion  of  the  appointment  functions.  Responses  in  the  "others" 
category  represent  general  comments  and  recommendations  that  did 
not  fit  into  one  of  the  eight  categories.  Table  16  presents 
comments  related  to  Improving  the  current  appointment  system. 


TABLE  16 


ADDITIONAL  COMMENTS  ON  HOW  TO  IMPROVE 
CURRENT  APPOINTMENT  SYSTEM 


ADDITIONAL  COMMENTS 

ABSOLUTE 

FREQ 

ADJUSTED 

FREQ 

(PCT) 

MORE  PHONE  LINES 

In-Hospital 

18 

7.6 

7.6 

Mail-Out 

55 

12.2 

12.2 

MORE  APPOINTMENT  CLERKS 

In-Hospital 

17 

7.2 

14.8 

Mail-Out 

41 

9.1 

21.3 

NO  LONG  DISTANCE  HOLD 

In-Hospital 

3 

1.3 

16.1 

Mail-Out 

19 

4.2 

25.5 
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Table  16  (Continued) 

ADJUSTED 

ABSOLUTE 

FREQ 

ADDITIONAL  COMMENTS 

FREQ 

(PCT) 

■Hi  31 

LESS  CONTACT  TIME 


In-Hospital 

19 

8.0 

24 . 1 

Mail-Out 

90 

20.0 

45.5 

NO  CHANGE 

In-Hospital 

92 

38.8 

62.9 

Mail-Out 

76 

16.9 

62.4 

MORE  DOCTORS 

In-Hospital 

2 

.8 

63.7 

Mail-Out 

9 

2.0 

64.4 

COMPLETELY  CENTRALIZE  SYSTEM 

In-Hospital 

10 

4.2 

67.9 

Mail-Out 

34 

7.6 

72.0 

COMPLETELY  DECENTRALIZE  SYSTEM 

In-Hospital 

0 

0 

67.9 

Mail-Out 

3 

0.6 

72.6 

OTHER 

In-Hospital 

76 

32.1 

100.0 

Mail-Out 

123 

27.4 

100.0 

Based  on  the  previously  stated  criteria  of  an  eighty  percent 
satisfaction  rate  in  order  to  determine  that  a  particular  area  is 
a  patient  satisfier,  an  evaluation  can  now  be  made  on  the  six 
areas.  The  data  presented  indicate  that  the  in-hospital  group 
was  satisfied  with  five  of  the  six  areas,  and  that  the  overall 
satisfaction  percentage  of  the  sixth  area  (79. 5f)  was  very  close 


48 


to  the  stated  criteria.  The  mall-out  group,  however,  expressed 
overall  satisfaction  in  two  of  the  six  areas,  both  of  which  dealt 
with  the  appointment  clerk  issues.  In  order  to  better 
understand  these  differences,  further  analysis  of  the  data  must 
be  accomplished. 

Comparing  Satisfaction  Levels  Between  Systems 
The  next  logical  step  in  the  analysis  of  the  data  is  to 
evaluate  the  patients'  satisfaction  with  the  two  types  of 
appointment  systems.  The  data  from  the  overall  opinion  question 
was  analyzed,  utilizing  Chi-square  to  determine  the  significance 
of  the  difference  between  the  centralized  and  decentralized 
systems.  Table  17  presents  the  overall  satisfaction  levels 
selected  by  users  of  the  centralized  and  decentralized  systems 
for  both  in-hospital  and  mail-out  surveys. 


TABLE  17 

LEVEL  OF  SATISFACTION  FOR  PATIENTS  USING  A 
DECENTRALIZED  &  CENTRALIZED  APPOINTMENT  SYSTEM 


IN-HOSPITAL 
ABSOLUTE  ADJUSTED 

FREQ  FREQ  (PCT) 

MAIL-OUT 

ABSOLUTE  ADJUSTED 
FREQ  FREQ  (PCT) 

VERY  DISSATISFIED 

Centralized 

8 

6.7 

42 

12.1 

Decentralized 

1 

0.7 

9 

7.1 

DISSATISFIED 


Centralized 

Decentralized 


15 

5 


12.5 

3.5 


57 

17 


16.4 

13.6 
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Table  17  (Continued) 


IN-HOSPITAL 
ABSOLUTE  ADJUSTED 

FREQ  FREQ  (PCT) 

MAIL-OUT 

ABSOLUTE  ADJUSTED 

FREQ  FREQ  (PCT) 

NEITHER 

Centralized 

21 

17.5 

45 

12.9 

Decentralized 

10 

7.0 

15 

11.9 

SATISFIED 

Centralized 

45 

37.5 

122 

35.1 

Decentralized 

53 

37.3 

40 

31.7 

VERY  SATISFIED 

Centralized 

31 

25.8 

82 

23.6 

Decentralized 

73 

51.4 

45 

35.7 

Chi-Square 

^  30 

.32878  Chi-Square  = 

8.051786 

P  =  • 

000004 

P 

=  .089699 

An  initial  review  of  the  in-hospital  results  suggests  that 
patients  using  a  decentralized  system  had  a  higher  overall 
opinion  of  the  patient  appointment  system.  Over  eighty-eight 
percent  of  the  in-hospital  group  who  used  a  decentralized  system 
expressed  overall  satisfaction  compared  to  63.3  percent  of  those 
using  the  centralized  system.  "Dissatisfied"  or  "Very 
Dissatisfied"  responses  were  given  by  approximately  nineteen 
percent  of  individuals  using  the  centralized  system,  compared  to 
approximately  four  percent  of  the  decentralized  system 


respondents  .  The  Chi-square  value  confirms  the  initial 
observation  that  there  is  a  significant  difference  between  the 
satisfaction  responses  of  the  two  groups  (p  <  .001),  although 
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caution  must  be  exercised  in  interpreting  this  finding  because 
one  of  the  cells  contains  less  than  five  observations. 

The  mail-out  group  results  appear  to  reflect  the  same 
tendency  toward  satisfaction  with  the  decentralized  system, 
however,  the  difference  is  not  as  pronounced.  Comparing  response 
percentages  over  the  five  levels  of  satisfaction  indicates  that 
there  is  little  difference  between  satisfaction  rates  of  groups 
using  centralized  and  decentralized  systems,  with  the  exception 
of  the  “very  satisfied"  level  where  the  decentralized  respondents 
exceeded  the  centralized  respondents  by  12.1  percent.  The  Chi- 
square  value  confirms  that  there  is  a  trend  toward  greater 
satisfaction  with  the  decentralized  system  (p  <  .10),  although 
this  difference  does  not  reach  the  criterion  .05  significance 
level . 

An  alternate  and  more  concise  method  of  measuring  the 
difference  in  patients'  satisfaction  with  the  two  types  of 
appointment  systems  is  to  compare  the  mean  satisfaction  scores  of 
each  group.  Mean  scores  for  each  group  were  computed  after 
assigning  a  numerical  value  to  each  respondent's  satisfaction 
level  (5  =  Very  Satisfied;  1  =  Very  Dissatisfied).  A  two-tail  t- 
test  was  used  to  compare  the  responses  to  the  overall  opinion 
question.  Table  18  presents  the  results  of  this  analysis. 
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TABLE  18 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  OVERALL  OPINION  OF  PATIENTS 
USING  A  CENTRALIZED  &  DECENTRALIZED  APPOINTMENT  SYSTEM 


CENTRALIZED 

DECENTRALIZED 

SURVEY  GROUP 

n 

Mean  SD 

Mean  SD 

In-Hospital 

120 

3.63  1.19 

142 

4.35  0.82 

T-Value  - 

-5.60 

2-Tall 

Probability  -  0.000 

Mail-Out 

348 

3.42  1.33 

126 

3.75  1.27 

T-Value  - 

-2.52 

2-Tail 

Probability  -  0.012 

The  in-hospltal  mean  satisfaction  score  for  those  individuals 
using  a  centralized  system  (3.63)  was  much  lower  than  for  those 
using  a  decentralized  system  (M.35).  The  two-tail  t-test  with 
separate  variance  estimate  confirms  that  there  is  a  significant 
difference  between  these  two  results  (p  <  .001).  The  mail-out 
group  results  do  not  differ  as  drastically  (3.42  compared  to 
3.75).  The  two-tail  t-test  does  support  that  there  is  a 
significant  difference  in  satisfaction  between  individuals 
utilizing  a  centralized  versus  a  decentralized  system  (p  <  .05). 

Even  though  the  general  opinion  question  provides  a  good 
comparison  of  satisfaction  levels  between  the  two  groups,  a  more 
thorough  analysis  of  the  data  is  required  to  understand  the 
reason  for  the  differences  in  satisfaction.  The  best  way  to  do 


this  is  to  analyze  responses  to  each  of  the  questions  about 
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satisfaction  with  specific  aspects  of  the  patient  appointment 
system.  Comparisons  between  respondents  using  centralized  and 
decentralized  systems  will  be  presented  for  each  question, 
beginning  with  the  area  which  had  the  largest  difference  in  mean 
satisfaction  scores.  Table  19  shows  the  comparison  of  mean 
satisfaction  responses  to  the  question  concerning  the  amount  of 
time  It  took  to  actually  make  the  appointment. 

TABLE  19 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  TIME  TO  MAKE  APPOINTMENT  FOR  PATIENTS 
USING  A  CENTRALIZED  &  DECENTRALIZED  APPOINTMENT  SYSTEM 


CENTRALIZED 

DECENTRALIZED 

SURVEY  GROUP 

n 

Mean 

SD 

n  Mean  SD 

In-Hospital 

122 

3.59 

1.21 

143  4.27  0.97 

T-Value  - 

-5.01 

2-Tail 

Probability  -  0.000 

Mail-Out 

3^6 

3.29 

1.42 

128  3.83  1.26 

T-Value  - 

-3.98 

2-Tail 

Probability  -  0.000 

The  In-hospltal  portion  of  the  table  Indicates  that  individuals 
using  the  decentralized  system  were  much  more  satisfied  with  the 
time  it  took  to  make  an  appointment  than  those  individuals  using 
the  centralized  system.  The  average  decentralized  satisfaction 
level  was  4.27,  while  the  centralized  mean  was  3.59.  The  two- 
tail  t-test  confirms  that  there  is  a  significant  difference 
between  these  two  groups  (p  <  .01).  A  like  comparison  of  the 
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mail-out  survey  responses  also  indicates  a  significant  difference 
favoring  the  decentralized  system  (p  <  .01). 

The  next  responses  to  be  compared  concern  the  issue  of 
satisfaction  with  the  amount  of  time  between  when  the  respondent 
made  the  appointment  and  the  actual  appointment  date  (referred  to 
as  lag  time).  Table  20  presents  comparison  of  the  mean 
satisfaction  scores  for  this  question. 

TABLE  20 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  APPOINTMENT  LAG  TIME  FOR  PATIENTS 
USING  A  CENTRALIZED  &  DECENTRALIZED  APPOINTMENT  SYSTEM 


SURVEY  GROUP 

CENTRALIZED 

n  Mean  SD 

DECENTRALIZED 

n  Mean  SD 

In-Hospital 

123 

3.85  1.15 

142  4.50  0.74 

T-Value  - 

-5.43 

2-Tail 

Probability  -  0.000 

Mail-Out 

343 

3.50  1.29 

128  3.80  1.16 

T-Value  - 

-2.57 

2-Tail 

Probability  -  0.011 

Respondents  from  the  In-hospltal  group  utilizing  the 
decentralized  appointment  system  had  a  mean  score  of  4.50 
compared  to  3.85  for  those  respondents  using  the  centralized 
system.  There  appears  to  be  a  significant  difference  in  these 
two  groups,  which  is  confirmed  by  the  two-tail  t-test  (p  <  .01). 
The  mail-out  respondents'  mean  scores  do  not  present  as  large  a 


difference  as  the  In-hospltal  group.  Those  individuals  utilizing 
a  decentralized  appointment  system  had  a  mean  score  of  3.80 
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compared  to  3.50  for  those  using  the  centralized  system.  The  t> 
test  confirms  that  this  difference  is  significant  (p  <  .05). 

Another  Issue  which  was  analyzed  was  the  respondents' 
satisfaction  with  the  time  and  date  of  the  appointment  they 
received.  Table  21  presents  the  comparison  of  the  mean 
satisfaction  scores  for  the  question  concerning  time  and  date  of 
appointment . 

TABLE  21 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  TIME  AND  DATE  OF  APPOINTMENT  OF  PATIENTS 
USING  A  CENTRALIZED  &  DECENTRALIZED  APPOINTMENT  SYSTEM 


CENTRALIZED 

DECENTRALIZED 

SURVEY  GROUP 

n 

Mean 

SD 

n 

Mean  SD 

In-Hospital 

120 

3.92 

1 .16 

143 

4.43  0.84 

T-Value  - 

-4.02 

2-Tail 

Probability  -  0.000 

Mail-Out 

347 

3.62 

1.25 

128 

3.85  1.29 

T-Value  - 

-1.77 

2-Tail 

Probability  -  0.078 

The  In-hospital  group  had  a  mean  score  of  3.92  for  the 
centralized  system  and  4.43  for  the  decentralized  system.  The 
results  of  the  two-tall  t-test  Indicate  that  there  is  a  signi¬ 
ficant  difference  between  these  two  groups  (p  <  .01).  In 
comparing  the  results  of  the  mail-out  respondents,  the  mean 
scores  of  the  respondents  using  the  centralized  and  decentralized 
systems  were  3.62  and  3.85,  respectively.  Even  though  there  is  a 
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difference  between  these  two  groups,  the  t-test  results  Indicate 
that  the  significance  of  this  difference  is  borderline  (p  <  .10). 

The  two  questions  concerning  information  provided  by  the 
appointment  clerk  and  treatment  by  the  appointment  clerk  had  mean 
response  scores  which  generally  do  not  appear  to  be  significantly 
different  for  the  two  appointment  systems.  Tables  22  and  23 
present  the  data  for  the  Issues  of  information  provided  by  the 
appointment  clerk  and  treatment  by  the  appointment  clerk, 
respectively . 

TABLE  22 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  INFORMATION  BY  THE  APPOINTMENT  CLERK  FOR  PATIENTS 
USING  A  CENTRALIZED  &  DECENTRALIZED  APPOINTMENT  SYSTEM 


CENTRALIZED 

DECENTRALIZED 

SURVEY  GROUP 

n 

Mean 

n  Mean  SD 

In-Hospital 

121 

4.27 

0.89 

141  4.57  0.60 

T-Value  - 

-3.08 

2-Tail 

Probability  -  0.002 

Mail-Out 

345 

4.23 

0.85 

127  4.25  0.75 

T-Value  - 

0.25 

2-Tail 

Probability  -  0.803 

Satisfaction  with  information  provided  by  the  appointment  clerk 
for  the  in-hospital  group  using  the  centralized  system  is 
represented  by  a  mean  score  of  4.27  compared  to  a  mean  score  of 
4.57  for  those  using  a  decentralized  system.  The  t-test  confirms 
that  this  difference  is  significant  (p  <  .01).  The  mail-out 
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groups'  response  to  the  same  question  resulted  In  a  mean  score  of 
4.23  for  the  centralized  system  and  4.25  for  the  decentralized 
system.  This  difference  was  not  significant  at  the  .05  level 

(p  >  .10). 


TABLE  23 

COMPARISON  OF  MEAN  SATISFACTION  SCORES 
FOR  TREATMENT  RECEIVED  FROM  APPOINTMENT  CLERK  FOR  PATIENTS 
USING  A  CENTRALIZED  A  DECENTRALIZED  APPOINTMENT  SYSTEM 


CENTRALIZED 

DECENTRALIZED 

SURVEY  GROUP 

Mean 

SD 

JQ 

Mean 

In-Hospltal 

122 

4.60 

0.64 

143 

4.61 

0.64 

T-Value  - 

-0.22 

2-Tall 

Probability  -  0 

.829 

Mall-Out 

347 

4.38 

0.77 

128 

4.33 

0.82 

T-Value  - 

0.57 

2-Tall 

Probability  -  0 

.570 

The  question  concerning  the  treatment  received  from  the 
appointment  clerk  resulted  In  a  mean  score  of  4.60  for  the 
cen'.rallzed  system  respondents  and  4.62  for  the  decentralized 
system  respondents  of  the  In-hospltal  survey  group.  The 
difference  between  these  two  groups  did  not  appear  to  be 
significant.  The  results  of  the  t*test  confirm  that  the 
Ifference  Is  not  significant  at  the  .05  level  (p  >  .10).  The 
.uail>out  groups'  response  to  this  question  resulted  In  a  mean 
score  of  4.38  for  those  Individuals  using  the  centralized  system 
and  4.34  for  the  decentralized  system  respondents.  This 
difference  also  Is  not  significant  at  the  .05  level. 


FOOTNOTES 


^Earl  R.  Babble,  Survey  Research  Methods.  Belmont, 
California:  Wadsworth,  1 983 ,  1 65  . 

^Kenneth  D.  Bailey,  Methods  of  Social  Research  2nd  ed. 
New  York:  The  Free  Press,  1982,  pp .  177-178. 

^Floyd  J.  Fowler,  Jr.,  Survey  Research  Methods.  London 
Sage  Publications,  1984,  p.  55. 

^Charles  H.  Backstrom  and  G.  Hursh-Cesar,  Survey 
Research,  New  York:  John  Wiley  and  Sons,  1981,  p.  236. 
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III.  CONCLUSIONS  AND  RECOMMENDATIONS 


The  specific  intent  of  this  research  project  was  to 
determine  if  there  is  a  significant  difference  in  patients'  level 
of  satisfaction  for  those  patients  utilizing  a  centralized 
versus  a  decentralized  system  for  scheduling  outpatient 
appointments  at  Madigan  Army  Medical  Center.  The  project  focused 
on  measuring  patients'  satisfaction  with  five  specific  areas  of 
the  outpatient  appointment  system.  The  sample  population  for 
this  research  consisted  of  two  major  groups:  (1)  patients  who 
were  using  the  outpatient  appointment  system,  and  (2)  retirees 
who  might  have  used  the  outpatient  appointment  system  in  the 
past.  The  large  survey  return  rate  indicates  that  the  patient 
appointment  system  is  a  topic  of  concern  for  the  MAMC  patient 
population . 

Findings 

This  study  found  that  there  is  a  significant  difference  in 
patients'  level  of  satisfaction  for  those  patients  utilizing  a 
centralized  versus  a  decentralized  appointment  system. 

Individuals  using  the  decentralized  system  were  more  satisfied 
with  the  following:  (1)  time  and  date  of  appointment;  (2)  time 
between  making  the  appointment  and  the  actual  appointment  date; 
(3)  the  amount  of  time  it  took  to  make  the  appointment;  and  (4) 
information  provided  by  the  appointment  clerk.  There  also  was  a 
significant  difference  in  satisfaction  for  the  overall  opinion 
question.  Those  individuals  using  a  decentralized  system  had  a 
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more  satisfied  overall  opinion  than  those  individuals  using  a 
centralized  system.  The  only  items  which  did  not  show  a 
significant  difference  in  satisfaction  levels  for  both  groups 
between  the  two  systems  were  (1)  the  information  provided  by  the 
appointment  clerk,  and  (2)  the  treatment  provided  by  the 
appointment  clerk.  A  significant  difference  did  exist  for 
satisfaction  with  the  information  provided  by  the  appointment 
clerk  within  the  in-hospital  group;  but  the  mail-out  group's 
responses  to  the  same  question  were  not  significantly  different. 

Comparing  the  responses  of  patients  using  a  centralized  and 
a  decentralized  system  indicates  that  the  items  which  were  less 
satisfying  were  those  which  related  to  availability  of  resources. 
For  both  groups  the  items  which  caused  greater  dissatisfaction 
were  the  same  items  which  have  been  addressed  by  individuals  who 
have  previously  studied  the  problems  of  the  MAMC  appointment 
system.  The  Memorandum  for  the  Deputy  Commander  for  Clinical 
Services  prepared  in  October,  198M  addresses  the  three  main 
problems  of  (1)  staffing,  (2)  communications  support,  and  (3) 
automation  support.  The  centralized  system  respondents'  greater 
dissatisfaction  with  the  four  areas,  including  (1)  time  and  date 
of  appointment,  (2)  amount  of  time  to  make  the  appointment,  (3) 
amount  of  time  between  making  the  appointment  and  the  actual 
appointment  date,  and  (4)  information  provided  by  the  appoint¬ 
ment  clerk,  can  be  attributed  to  three  major  resource  problems: 
(1)  lack  of  adequate  numbers  of  appointment  clerks,  (2)  poor 
communications  equipment,  and  (3)  inadequate  automation  support. 
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The  responses  of  the  decentralized  group  also  indicated  that 
these  items  were  a  problem;  however,  it  appears  that  the 
magnitude  of  this  problem  is  less  in  the  decentralized 
appointment  system.  The  Impression  that  dissatisfaction  with  the 
appointment  system  can  be  traced  to  inadequate  resources  is  also 
supported  by  the  comments  presented  on  how  to  improve  the  current 
appointment  system.  The  four  most  frequent  suggestions  for  both 
groups  were  (1)  more  phone  lines,  (2)  more  appointment  clerks, 

(3)  reduce  contact  time  with  appointment  clerk,  and  (4)  eliminate 
the  practice  of  putting  long  distance  callers  on  hold. 

Although  significant  differences  were  found  between 
centralized  and  decentralized  appointment  users  in  both  in- 
hospital  and  mail-out  surveys,  the  in-hospital  group  as  a  whole 
tended  to  be  more  satisfied.  More  than  eighty  percent  of  the  in- 
hospital  survey  group  expressed  satisfaction  with  five  or  six 
areas  of  the  appointment  system  they  used.  In  the  sixth  area, 
overall  opinion,  79.5  percent  of  the  in-hospital  group  expressed 
satisfaction,  which  closely  approximates  the  eighty  percent  HSC 
criteria.  More  than  eighty  percent  of  the  mail-out  group 
expressed  satisfaction  with  only  two  of  the  six  aspects  of  the 
appointment  system.  These  two  areas  were  (1)  treatment  provided 
by  the  appointment  clerk,  and  (2)  information  provided  by  the 
appointment  clerk.  Approximately  sixty  percent  of  the  mail-out 
group  expressed  satisfaction  with  the  other  four  areas.  It 
appears  that  the  mall-out  group  was  more  dissatisfied  than  the 
In-hospltal  group.  This  difference  between  the  two  groups  might 
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be  explained  by  the  In-hospltal  group  being  "in-the-system"  while 
the  mail-out  group  was  at  home,  trying  to  enter  the  system. 

Since  the  mail-out  group  consisted  of  retirees,  their 
dissatisfaction  with  difficulty  accessing  the  system  because  of 
their  priority  in  the  queue  of  beneficiaries  could  have  biased 
their  responses  to  the  questions  about  the  appointment  system. 

Considerations 

Several  considerations  should  be  addressed  before  any  final 
recommendations  about  modifying  MAMC's  current  appointment  system 
can  be  made.  First,  consideration  should  be  given  to  the 
reliability  of  the  Information  given  by  survey  respondents. 

Since  all  survey  participants  were  asked  to  base  their  answers  on 
their  most  recent  outpatient  appointment  experience,  the  mail-out 
group's  answers  might  contain  some  invalid  information.  The 
respondents  could  have  had  some  recall  loss  about  their  last 
appointment.  The  in-hospital  group's  responses  should  have  been 
fairly  accurate  since  they  were  physically  present  in  the 
hospital  waiting  for  the  appointment  about  which  they  were 
queried.  Also  Included  in  this  concern  is  whether  or  not  the 
patient  remembered  if  they  called  the  PAS  or  the  clinic  to  make 
their  appointment.  It  might  be  possible  that  some  respondents 
did  not  know  the  difference  between  calling  the  clinic  and 
calling  the  PAS. 

This  research  study  addressed  the  issue  of  patient  satis¬ 
faction  and  does  not  consider  the  cost  factors  of  operating  a 
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totally  centralized  or  decentralized  appointment  system.  Even 
though  patient  satisfaction  is  an  important  issue,  the  resource 
costs  for  totally  centralizing  or  decentralizing  the  appointment 
system  must  be  a  major  consideration  before  any  changes  are 
initiated.  If  the  decision  is  made  to  totally  decentralize  the 
appointment  function  without  further  study  into  the  resource 
allocations  necessary  to  complete  such  a  task,  then  an  under¬ 
resourced,  decentralized  system  might  cause  greater  dissatis¬ 
faction  than  the  currently  established  system. 

Another  consideration  is  whether  or  not  a  decentralized 
system  is  better  for  everyone  concerned.  This  study  addresses 
only  patient  satisfaction  and  not  the  issue  of  satisfaction  of 
the  MAMC  staff.  Even  though  this  study  confirms  that  patients 
are  more  satisfied  with  a  decentralized  system,  the  staff's 
satisfaction  with  the  appointment  system  must  also  be  considered 
before  any  changes  are  made.  Changing  the  appointment  system  at 
MAMC  without  staff  input  could  cause  staff  resentment  and 
resource  shortages  which  would  decrease  system  efficiency. 

A  major  change  in  the  MAMC  appointment  system  may  not  be 
necessary  to  increase  patient  satisfaction.  Minor  changes  in  the 
current  centralized  system  could  be  considered  in  order  to 
improve  its  image  with  patients.  Further  consideration  could 
also  be  given  to  improving  those  areas  that  are  currently 
"patient  dissatisf iers. "  As  previously  stated,  these  areas  are 
resource  related.  Cost  versus  benefits  of  any  major  changes  must 
be  carefully  weighed.  Any  proposal  should  be  carefully 
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implemented  and  evaluated  for  effectiveness,  improvement  in 
efficiency,  and  impact  on  patient  satisfaction. 

Recommendations 

A  total  decentralization  of  the  appointment  system  should 
not  be  accomplished  based  solely  on  the  results  of  this  research. 
The  patients  expressed  more  satisfaction  with  the  decentralized 
appointment  system  as  it  currently  exists;  however,  no  additional 
personnel,  phone  lines,  and  automation  resources  are  currently 
available  to  totally  decentralize  the  appointment  system.  If  the 
entire  system  was  decentralized  without  such  additional 
resources,  it  would  be  difficult  to  meet  the  appointment  requests 
in  each  individual  clinic.  The  decentralized  mode  could  become 
the  less  favorable  method  of  making  appointments. 

In  light  of  the  findings  and  considerations  previously 
presented,  several  recommendations  for  further  study  are 
appropriate.  The  current  appointment  system  at  MAMC  must  be 
further  analyzed  to  determine  the  best  method  for  scheduling 
appointments.  The  results  of  this  research  should  be 
supplemented  by  further  study  to  determine  the  staff's 
perceptions  of  the  system,  the  affordability  and  cost 
effectiveness  of  any  changes,  and  the  priority  of  appointment 
system  change  in  relationship  to  the  other  major  management  and 
resource  problems  currently  being  addressed  by  the  command. 

If,  after  additional  research  and  study,  a  decision  is  made 
to  change  the  existing  appointment  system,  a  pilot  study  should 
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be  conducted  to  evaluate  the  resultant  changes  In  patient 
satisfaction,  staff  satisfaction,  cost  effectiveness,  and  ability 
to  meet  the  mission  of  providing  quality  patient  care  to 
beneficiaries.  A  pilot  study  would  provide  an  evaluation  of  the 
effectiveness  of  the  proposed  changes  before  costly  large-scale 
changes  are  made. 

In  a  healthcare  setting  heavily  weighted  towards  ambulatory 
care,  the  issue  of  petient  scheduling  is  extremely  important. 
Appointment  systems  must  be  carefully  planned  and  executed  in 
order  to  maximize  efficiency,  effectiveness,  and  patient 
satisfaction . 


APPENDIX  A 


MEMORANDUM  FOR  DEPUTY  COMMANDER  FOR 
CLINICAL  SERVICES  REFERENCE 
PATIENT  APPOINTMENT  SYSTEM  (PAS) 
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HCPLV  TO 
ATTENTION  OF 


HSKJ-CL 


DEPARTMENT  OF  THE  ARMY 
HEADQUARTERS  MAOIGAN  ARMY  MEDICAL  CENTER 
TACOMA.  WASHINGTON  98431 


17  October  1984 


MEMDRAlSirfjM  EXDR  DEPUTY  COyMANDER  EXDR  CLIMICAL  SERVICES,  MAMC 

SUBJECT:  Problems  Associated  with  the  Patient  Appoint  System  (PAS)  at 
Madigan  Army  Medical  Center 


1.  References: 

a.  Ambulatory  Patient  Care  Program  Document,  Chapter  3E  (May  1981)  . 

b.  Antiulatory  Patient  Care  Model  #1  (July  1974) . 

c.  Health  Services  Command  Parrphlet  40-7-1  (Draft)  (April  1984) . 

2.  Purpose:  To  provide  information  concerning  operational  difficulties 
currently  being  experienced  with  the  Madigan  Axn^'  Medical  Center's  Patient 
Appointment  System  (PAS)  and  to  identify  resource  requirements  and  methodologies 
that  may  minimize  these  difficulties. 

3.  Discussion:  The  PAS  currently  used  at  MAMC,  has  been  operational  since 
N^y  1982.  It  provides  support  to  nineteen  distinct  clinics.  The  PAS  consists 
of  1)  a  ocitputer  operated  program,  maintained  on  a  Burroughs  B1865;  2)  a  staff 
consisting  of  one  svjpervisor,  one  lead  appointment  clerk,  and  eight  appointment 
clerks;  and  3)  a  ccimiunication  system  consisting  of  eight  inccming  telephone 
lines  which  are  distributed  through  the  use  of  an  Automatic  Call  Distributions 
System  (ACDS) .  The  ACDS  is  capable  of  "stacking"  or  holding  fifteen  inccming 
telephone  calls  for  this  distribution  process.  (Dperationally,  the  ACDS  accepts 
an  inccming  telephone  call,  executes  a  short  progranmed  message,  and  places  or 
"stacks"  the  caller  on  hold  for  the  next  available  appointment  clerk.  The 
appointment  clerk  ansvi^rs  the  telephone,  verifies  selected  patient  registration 
information,  and  negotiates  an  appointment  for  the  patient  based  on  patient 
desires/requirements  arid  practitioner  availability.  This  process  requires  the 
corputer  program  to  furnish  four  separate  screens  of  information  to  the 
appointment  clerk.  In  addition,  appxjintment  clerks  also  perform  secondary  roles 
of  di^ensing  information  relating  to  projected  practitioner  availability, 
furnishes  procedures  the  p>atient  must  psarticipate  in  to  gain  entry  to  limited 
specialty  services,  provides  limited  information  regarding  CHAMPUS  (referral 
function) ,  the  hours  of  different  c  3inic  operations ,  accessibility  bo  spjecif ic 
clinic  services  by  pjatient  category,  and  responses  to  a  multitude  of  other 
similar  information  demands.  In  selected  cases  the  appointment  clerks  may 
paxivide  a  screening  function  in  order  to  direct  patients  to  the  correct  clinical 
service.  These  secondary  roles  are  of  significant  iitportance  to  the  fcicility  and 
the  p>atient  cadler.  However,  PAS  cannot  meet  its  primary  role  without  the 
availability  of  adequate  clinic  appointments  in  relationship  to  patient  needs. 
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HSKJ-CL  17  October  1984 

SUBJECT:  ProblOTs  Associated  with  the  Patient  Appointment  System  (PAS)  at 
Madigan  Arny  Medical  Center 

Accordingly,  a  well  equiped,  well  staffed,  and  efficiently  managed  PAS  cannot 
meet  all  parent  requirements  at  all  times  unless  practitioner  availability  exceeds 
patient  dentand  in  each  clinical  service.  Recently,  the  hospital  has  been  the 
recipient  of  a  significant  increase  in  patient  ccrplaints  and  inquiries  relating 
to  limited  accessibility  to  the  PAS.  This  increase  correlates  with  a  documented 
surge  in  patient  demands  placed  on  the  eight  trunk  telephone  switches  established 
for  PAS  by  the  U.S.  Army  Ccntnunications  Ccnmand. 

Specific  difficulties  concerning  each  aspect  of  the  PAS  are  discussed  further  under 
the  following  issues: 

a.  Staffing  Requirements.  In  March  1984  a  U.S.  Army  Health  Services  Cctnmand 
sponsored  manpower  survey  established  a  recognized  personnel  requirement  for  the 
PAS  operation  consisting  of  one  supervisor,  one  lead  appointment  clerk,  and  nine 
appointment  clerks,  A  copy  of  the  survey  document  is  attached  (Atch  1) .  The  most 
significant  aspect  of  this  document  is  that  the  manpower  requirCTients  are  based  on 
acccrplished  workload  (patient  contacts) ,  without  regard  to  potential  patient  danand. 

In  response  to  these  requirements,  the  PAS  has  an  authorized  staff  of  one  supervisor, 
one  lead  appointment  rlerk,  and  eight  appointment  clerks.  One  of  the  appointment 
clerks  has  been  in  a  Li-^OP  status  since  July  1984  awaiting  a  possible  medical 
retirement  determination.  Thus,  the  PAS  has  a  current  staff  of  nine  full-time 
tsiployees.  This  staffing  level  is  incapable  of  meeting  t.he  danands  of  the  patient 
population  in  its  present  configuration  despite  e-xceeding  e:<pected  individual 
appointment  clerk  productivity  as  established  in  reference  lb,  preceding  page.  In 
this  regard,  patient  demand  identified  by  USACC  telephonic  monitoring  instruments 
indicates  inconing  telephone  inquiries  far  in  excess  of  PAS  capability.  These 
inquiry  rates  will  be  further  discussed  in  the  comnunication  system  portion  of  this 
d'^cument  but  at  the  present  time  the  PAS  is  able  to  respond  to  only  one-half  of  the 
inconing  workload.  In  addition,  support  for  administrative  details  such  as  processing 
the  master  schedule ,  reproduction  of  schedules ,  handling  patient  cancellations ,  etc . , 
rust  be  taken  fron  manpower  resources  that  should  be  devoted  to  receiving  inconing 
telephone  ccntacts. 

b.  CoTTnunications  System.  There  are  a  few  problems  associated  with  this 
portion  of  the  PAS  process.  However,  these  problems  are  readily  identifiable  and 
per  conversation  with  the  Commander,  U.S.  Army  Ccmunications  Ccmand,  will  be 
rectified  at  the  earliest  opportunity.  The  first  problan  with  the  system  is  the  limited 
capability  of  the  recorded  patient  message.  Attonpts  to  extend  the  capability  of 

the  recorded  message  in  terms  of  length  and  message  changeability  have  been 
unsuccessful  (Atch  2) .  This  precludes  a  fiilly  functional  instrument  that  has  the 
potential  to  respond  to  patient  inquiries,  perhaps  without  the  utilization  of  an 
appointment  clerk.  Second,  a  dedicated  line  does  not  exist  for  the  receipt  of 
appointment  cancel laticxis  on  a  recorded  basis.  This  lack  of  capability  discourages 
a  patient  from  CcLLling  the  PAS  to  cancel  an  appointment  and  produces  higher  "no  shew" 
rates  and  resultant  opportunity  costs. 

The  major  iitpact  of  the  comunication  systan  is  directed  at  the  total  Fort  Lewis 
telephone  systan.  According  to  the  USACC  there  are  122  telephone  lines  coming 
into  the  Post.  Eight  of  these  lines  are  dedicated  to  the  PAS  and  114  lines 
vfeoignat:*^  for  other  post  activities.  A  blocking  device  exists  at  the  main  post  . 
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stvitch  with  the  function  of  separating  the  two  systans.  Unfortunately,  the 
eight  lines  for  PAS  receive  telephone  calls  far  in  excess  of  capability.  This 
causes  a  significant  block  in  the  system  and,  as  a  result,  has  tied  up  telephone 
switches  in  the  local  area.  USACC  has  been  informed  that  as  a  result  of  this 
tie-up  the  separating  device  at  the  main  Fort  Lewis  switch  may  be  ranoved  to 
increase  lines  for  incoming  telephone  calls.  This  action  will  lessen  the  impact 
on  local  areas  but  is  predicted  to  block  the  use  of  the  total  post  telephone 
system.  The  number  of  inccming  calls  to  PAS  cannot  be  understated.  Recent 
information  provided  by  USACC  indicates  that  PAS  has  e.xceeded  40,000  telephone 
calls  in  a  single  8  hour  day.  The  dramatic  increase  in  telephone  calls  is 
re-].ected,  for  similar  periods,  over  a  twelve  month  period  on  the  attached 
report  furnished  by  USACC  (Atch  3) .  Fortunately,  these  telephone  calls  do  not 
indicate  the  number  of  callers.  A  stxrvey  conducted  by  the  PAS  indicates  that  an 
average  patient  makes  16  atteirpts  to  contact  the  PAS  before  the  telephone  call  is 
answered.  Thus,  40,000  telephone  calls  represent  approximately  2,500  patients. 
Discounting  holdovers  that  were  unable  to  access  PAS  on  previous  days,  the  total 
number  of  callers  is  estimated  at  1500-1600  per  day.  .At  the  present  tire  the 
PAS  staff  is  able  to  respond  to  1000  calls  per  day. 

c.  Autcmation  Support.  The  Burroughs  B-1865  has  a  memory  capacity  of  only  one 
megabyte.  In  relationship  to  this  memory,  the  PAS  must  actively  corpete  with  other 
hospital  activities  such  as  Personnel  Division,  Nutrition  Care  Division,  Logistics 
Division,  Patient  Administration  Division,  and  the  Dept  of  Nursing  for  systoris 
access.  At  the  present  tire,  the  PAS,  as  a  single  entity,  utilizes  approximately 
80-90  percent  of  all  available  marory.  As  a  result  the  system  consistently  runs: 
out  of  msnory,  causing  a  systan  shutdown  or  a  delay  in  response  tire  for  the  four 
appointment  panels  required  to  consunmate  each  patient  appointment.  In  addition 
to  an  insufficient  .memory  capability,  the  soft^vare  of  the  PAS  has  considerable 
documented  design  flaws.  The  Autcmation  Management  Office,  Health  Services 
Cesnnand,  has  retained  Assigned  Respansible  Agency  (ARA)  authority  to  correct  these 
design  flaws  based  on  input  from  PAS.  These  design  flaws  are  sutmitted  as 
Engineering  Change  Propasails  (ECP)  for  correction.  There  are  two  major  problems 
associated  with  this  process.  First,  there  is  a  significant  tire  lapse  for 
corrective  action  by  HSC.  The  AMO  at  HSC  has  estimated  current  ECP's  will  take 
three  man-years  by  a  program  anailyst  to  correct.  A  copy  of  the  outstanding  ECP's 
is  attached  at  Atch  4. 

w -oond,  the  AMD  at  HSC  is  unable  to  adequately  test  design  flaw  psackages  once  they 
are  corrected  because  the  systan  and  patients  are  located  at  MAMC.  As  a  result 
of  this  systan,  approximately  30  percent  of  all  supposedly  corrected  design  flaws 
must  be  returned  to  HSC  for  additional  corrective  action.  Discrepsancies  in  the 
automation  support  system  have  been  well  documented  cr/er  the  preceding  two  years. 
This  concern  is  well  expressed  in  a  letter  sent  to  H.SC  by  the  Connanding  General, 
MAMC,  in  June  1984  (Atch  5) . 

4.  Reconnended  Actions:  In  response  to  these  difficulties,  a  number  of  actions 
would  apjpsear  to  be  appropriate  at  this  time  if  the  PAS  is  to  function  in  an 
efficient  and  effective  manner.  These  reoanrended  actions  are  detailed  in  the 
following  categories : 

/ 
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a.  Staffing  Requirements.  It  is  reconmended  that  the  PAS  be  staffed  at 
100  perc^t  of  its  recognized  requirements.  This  will  permit  full  use  of  the 
existing  terminals  in  PAS  and  minimize  administrative  functions  currently  being 
corpleted  by  the  c^jpointment  clerks.  In  addition ,  authority  should  be 
granted  to  permit  the  hiring  of  four  part-time  enployees  to  extend  PAS  operations 
from  1630  to  2030  hours.  This  wuld  permit  operation  of  the  PAS  from  0730-2030 
hours  each  work  day.  The  benefits  of  this  adjustment  would  be  two-fold  in  that 
sufficient  manpower  would  be  available  to  handle  between  1500-2000  patient 
inquiries  per  day  and  at  least  four  hours  of  workload  woiold  be  placed  on  the 
ccmputer  during  periods  of  least  ocrpeting  interests.  Supplemental  terTninal 
equipment,  additional  telephone  lines,  and  other  office  equipment  woxiLd  not  be 
required  to  irrplement  this  staffing  package.  It  is  also  reccrmended  that  the 
hiring  of  part-time  enployees  and  expansion  of  operating  hours  be  enacted  on  a 
six  month  trial  beisis  with  a  continuance  evaluation  conducted  at  the  end  of  the 
period. 


b.  Conmunication  Systen.  The  efficacy  of  the  ccmmunication  system  is 
directly  related  to  the  staffing  corputer  capability  of  PAS.  Thus,  increased 
staffing  and  an  inproved  conputer  system  will  relieve  numerous  difficulties  with 
the  teleptone  s^^sten.  However,  it  is  felt  that  the  expansion  of  the  recorded 
message  capability  and  a  dedicated  capacity  for  patient  cancellation  calls  are 
imperative  as  a  means  of  improving  the  system.  In  addition,  the  Cormander, 

USACC,  has  recottrended  the  conduct  of  a  review  and  evaluation  of  the  corrmriication 
system  by  outside  expertise.  In  this  regard,  USACC  will  initiate  action  to  conduct 
the  review  and  evaluation  within  the  next  fourteen  days. 

c.  Automation  Support.  The  present  ccmputer  system  has  a  significant  inpact 
on  the  difficulties  being  experienced  by  PAS.  The  lack  of  sufficient  ccrputer 
mamory,  with  its  resultant  slow-response  time  and  periodic  system  shutdown, 

and  the  hospital's  inability  to  provide  speedy  cn-site  corrections  of  recognized 
'.esign  flaws  in  the  systen  are  a  major  source  of  our  limited  ability  to  respond 
to  the  patient  population.  Accordingly,  the  following  recommendations  should  be 
enacted: 


(1)  The  inmediate  acquisition  and  installation  of  a  Burroughs  1955 
ccrputer.  This  corputer  would  provide  the  PAS  sufficient  nonory  for  operations 
and  also  increase  response  time  and  alleviate  periodic  system  shutdowns 
experienced  with  the  present  system.  There  is  a  B-1955  available  frcm  the  Defense 
Mapping  Agency;  hcwever,  cis  of  this  date  it  has  not  been  identified  for  transfer 
to  MAMC  despite  numerous  conversations  and  requests  submitted  to  H^. 

(2)  ARA  authority  should  be  transferred  frcm  HSC  to  MAMC  for  the 
correction  of  design  flaws  on-site  at  the  user  facility.  This  transfer  of  function 
would  psermit  quicker  system  correct' on  and  an  ability  to  test  the  system  changes 
prior  to  inplementation.  In  conjunction  with  this  transfer,  the  Autcmation 
Management  Office,  MAMC,  vould  require  two  additional  program  analysts  to  correct 
the  existing  design  flat 
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5.  The  three  factors  that  inpact  on  the  operation  of  an  efficient  and  effective 
PAS  have  been  discussed  in  detail.  These  factors  are  significantly  inter¬ 
related  and.  must  be  resolved  cis  a  package  if  the  systan  is  to  accorplish  its 
primary  role  of  providing  clinic  appointment  periods  to  the  eligible  beneficiaries 
that  utilize  the  facility  and  its  secondary  role  of  service  as  an  information 
center  for  our  patient  popiUation. 


5  Atchs  ROBERT  F.  RJRPHY 

LTC,  MSC 

Admin  Coordinator 

Deputy  Catmander  for  Clinical  Services 
Madigan  Army  Medical  Center 
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Dear  Patient: 

A  study  is  being  conducted  to  determine  how  you  feel  about 
the  system  used  for  making  clinic  appointments  at  Madlgan  Army 
Medical  Center  (HAMC).  You  are  being  asked  to  complete  certain 
portions  of  the  attached  questionnaire  in  order  that  the  system 
may  be  more  responsive  to  your  needs.  Your  answer  should  be 
based  on  your  experience  making  the  appointment  for  today's  visit 
and  not  on  any  other  experiences  you  have  had  with  other 
appointment  systems.  Your  answers  will  be  combined  with  those  of 
other  patients  and  presented  anonymously  to  the  MAMC  Commander  at 
the  completion  of  the  survey. 

Please  be  open  and  honest  about  your  experiences.  This 
questionnaire  cannot  be  connected  with  you  or  your  physician 
since  you  are  not  being  asked  to  orovlde  specific  information 
about  yourself. 

Your  cooperation  In  completing  the  applicable  portions  of 
the  questionnaire  will  be  greatly  appreciated  and  will  provide 
valuable  Information  which  may  be  used  to  make  the  outpatient 
appointment  system  serve  you  better. 


DARRYL  H.  POWELL  M.D. 

BG,  MC 

Commanding 


PLEASE  TORN  TO  PAGE  1  AND  FOLLOW  THE  INSTRUCTIONS  FOR  COMPLETING 
EACH  QUESTION. 
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1 .  Please  place  an  X  In  the  box  in  front  of  the  statement  which 
best  describes  your  current  status. 

[]  Active  Duty  Service  Member 

[]  Family  Member  of  Active  Duty  Service  Member 
[]  Retired  Service  Member 

[]  Family  Member  of  Retired  Service  Member 
[]  Family  Member  of  Deceased  Service  Member 
[ ]  Other 


2.  How  long  have  you  used  the  outpatient  clinic  services  at 
MAMC? 


[] 

Less 

than  6 

months 

[] 

6  months  to 

11  months 

[] 

1  to 

3  years 

[] 

More 

than  3 

years 

3.  What  method  best  described  how  you  made  your  appointment  for 
today’s  visit? 

[]  Called  the  Central  Appointment  System.  (Please  turn  to 
page  2  and  answer  only  those  questions  on  page  2) 

[]  Called  this  clinic  for  an  appointment.  (Please  turn  to 
page  3  and  answer  only  those  questions  on  page  3) 

[]  Made  appointment  In  person  at  this  clinic.  (Please  turn 
to  page  3  and  answer  only  those  questions  on  page  3) 

C]  Appointment  was  made  by  another  method.  (Please  turn  to 
page  4  and  answer  only  the  questions  on  page  4) 


Page  1 


Please  circle  the  number  which  best  describes  your  feelings  about  each  of  the 
following  Issues  related  to  the  way  In  which  you  made  today's  appointment. 

Higher  numbers  Indicate  satisfaction  and  lower  numbers  Indicate  dissatisfaction. 
Please  consider  only  today’s  experience  at  MAMC  and  not  any  other  time  or 
appointment  system. 


VERY 

SATISFIED 

SATISFIED 

NEITHER 

SATISFIED/ 

DISSATISFIED  DISSATISFIED 

VERY 

DISSATISFIED 

The  amount  of 
time  It  took  you 
to  make  the  appt. 

5 

M 

3 

2 

1 

The  way  you  were 
treated  by  the 
appt  clerk. 

5 

4 

3 

2 

1 

Being  able  to  get 
your  appt  at  the 
time  and  date  you 
desired . 

5 

4 

3 

2 

1 

The  amount  of  time  5 

between  when  you 
made  the  appt  and 
the  actual  appt  date. 

4 

3 

2 

1 

Information  given 
to  you  by  the  appt 
clerk  with  regard 
to  the  appt. 

5 

4 

3 

2 

1 

Your  overall 
opinion  of  the 
Central  Appt 

System  at  MAMC. 

5 

4 

3 

2 

1 

If  you  can  change  anything 

about  the 

Central  Appointment 

System , 

what  would 

it  be? 


PLEASE  TURN  TO  PAGE  5  FOR  INSTRUCTIONS  ON  TURNING  IN  YOUR  SURVEY. 
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Appointment  Made  by  Contacting  Clinic 

Please  circle  the  number  which  best  describes  your  feelings  about  each  of  the 
following  Issues  related  to  the  way  In  which  you  made  today's  appointment. 

Higher  numbers  Indicate  satisfaction  and  lower  numbers  Indicate  dissatisfaction. 
Please  consider  only  today's  experience  at  MAMC  and  not  any  other  time  or 
appointment  system. 


VERY 

NEITHER 

SATISFIED/ 

VERY 

SATISFIED 

SATISFIED 

DISSATISFIED 

DISSATISFIED 

DISSATISFIED 

The  amount  of  time  5 

time  It  took  you 
to  make  the  appt. 

4 

3 

2 

1 

The  way  you  were  5 

treated  by  che 
appt  clerk. 

4 

3 

2 

1 

Being  able  to  get  5 

your  appt  at  the 
tlm )  and  date  you 
desired . 

4 

3 

2 

1 

The  amount  of  time  5 

between  when  you 
made  the  appt  and 
the  actual  appt  date. 

4 

3 

2 

1 

Information  given  5 

to  you  by  the  appt 
clerk  with  regard 
to  the  appt. 

4 

3 

2 

1 

Your  overall  5 

opinion  of  this 
clinic's  appt 
system. 

4 

3 

2 

1 

7.  If  you  can  change  anything  about  this  clinic'  s  appointment  system,  what 
would  It  be? 


PLEASE  TORN  TO  PAGE  5  FOR  INSTRDCTIONS  ON  TURNING  IN  YOUR  SURVEY. 
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Other  Appointment  Systems 


If  you  did  not  make  today's  appointment  by  contacting  the  Central 
Appointment  System  or  by  contacting  the  clinic,  please  describe 
below  how  you  obtained  your  appointment. 


PLEASE  TURN  TO  PAGE  5  FOR  INSTRUCTIONS  ON  TURNING  IN  YOUR  SURVEY. 


Page 


I 


77 


Turn  In  Instructions 

Please  place  your  completed  survey  in  the  attached  envelope,  seal 
it,  and  return  it  to  the  clinic  receptionist.  The  receptionist 
will  not  open  the  envelope  and  will  Insure  that  the  survey  is 
returned  to  the  surveyor. 


THANK  YOU  FOR  YOUR  TIME  IN 
COMPLETING  THIS  SURVEY 


Page  5 
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Dear  Retiree: 

A  study  is  being  conducted  to  determine  how  you  feel  about 
the  system  used  for  making  clinic  appointments  at  Madlgan  Army 
Medical  Center  (MAMC) .  This  study  will  help  to  Improve  the 

appointment  systems  at  MAMC.  You  have  been  selected  as  part  of  a 
systematic  sample.  Please  take  the  five  minutes  required  to 
complete  this  survey.  Your  answer  should  be  based  on  your 

experience  making  the  appointment  the  last  time  you  visited  MAMC 

and  not  on  any  other  experiences  you  have  had  with  other 

appointment  systems.  Your  answers  will  be  combined  with  those  of 
others  surveyed  and  presented  anonymously  to  the  MAMC  Commander 
at  the  completion  of  the  survey. 

Please  be  open  and  honest  about  your  experiences.  This 
questionnaire  cannot  be  connected  with  you  since  you  are  not 
being  asked  to  provide  specific  information  about  yourself. 


DARRYL  H.  POWELL  M.D. 

BG,  MC 

Commanding 
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1.  Please  place  an  X  in  the  box  In  front  of  the  statement  which 
best  describes  your  current  status. 


[]  Active  Duty  Service  Member 

[]  Family  Member  of  Active  Duty  Service  Member 
[]  Retired  Service  Member 

[]  Family  Member  of  Retired  Service  Member 
[]  Family  Member  of  Deceased  Service  Member 
[]  Other 

2.  How  long  have  you  used  the  outpatient  clinic  services  at 
MAMC? 

[]  Less  than  6  months 
[]  6  months  to  11  months 
[]  1  to  3  years 
[]  More  than  3  years 

[]  I  have  never  used  the  outpatient  services  at  MAMC(PLEASE 
TORN  TO  PAGE  5  AND  COMPLETE  THE  QUESTIONS  ON  THAT  PAGE) 

3.  Nhat  method  best  described  bow  you  made  your  appointment  for 
your  most  recent  visit  to  MAMC? 

[]  Called  the  Central  Appointment  System.  (Please  turn  to 
page  2  and  answer  only  those  questions  on  page  2) 

[]  Called  the  respective  clinic  for  an  appointment. 
(Please  turn  to  page  3  and  answer  only  those  questions  on 
page  3) 

[]  Made  appointment  in  person  at  tb^  spectlve  clinic 
(Please  turn  to  page  3  and  answer  o  nose  questions  on 

page  3) 

[]  Appointment  was  made  by  another  method.  (Please  turn  to 
page  4  and  answer  only  the  questions  on  page  4) 


Page  1 
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Appointment  Made  by  Calling  Central  Appointment  System 

Please  circle  the  number  which  best  describes  your  feelings  about  each  of  the 
following  Issues  related  to  the  way  In  which  you  made  your  most  recent 
appointment.  Higher  numbers  Indicate  satisfaction  and  lower  numbers  Indicate 
dissatisfaction.  Please  consider  only  today's  experience  at  MAMC  and  not  any 
other  time  or  appointment  system. 


NEITHER 

VERY  SATISFIED/  VERY 

SATISFIED  SATISFIED  DISSATISFIED  DISSATISFIED  DISSATISFIED 


1.  The  amount  of 
time  It  took  you 
to  make  the  appt. 

2.  The  way  you  were 
treated  by  the 
appt  clerk. 

3.  Being  able  to  get 
your  appt  at  the 
time  and  date  you 
desired. 

4.  The  amount  of  time 
between  when  you 
made  the  appt  and 
the  actual  appt  date. 

5.  Information  given  5^3  2  1 

to  you  by  the  appt 

clerk  with  regard 
to  the  appt. 

6.  Your  overall  543  2  1 

opinion  of  the 

Central  Appt 
System  at  MAMC. 

7.  If  you  can  change  anything  about  the  Central  Appointment  System,  what  would 
It  be? 


PLEASE  TURN  TO  PAGE  6  FOR  INSTRUCTIONS  ON  TURNING  IN  YOUR  SURVEY. 
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Appointment  Made  by  Contacting  Clinic 

Please  circle  the  number  which  best  describes  your  feelings  about  each  of  the 
following  Issues  related  to  the  way  in  which  you  made  your  most  recent 
appointment.  Higher  numbers  Indicate  satisfaction  and  lower  numbers  Indicate 
dissatisfaction.  Please  consider  only  today's  experience  at  MANC  and  cot  any 
other  time  or  appointment  system. 

NEITHER 

VERY  SATISFIED/  VERY 

SATISFIED  SATISFIED  DISSATISFIED  DISSATISFIED  DISSATISFIED 


1 .  The  amount  of  time  5 

time  It  took  you 

to  make  the  appt. 

2.  The  way  you  were  5 

treated  by  the 

appt  clerk. 

3.  Being  able  to  get  5 

your  appt  at  the 

time  and  date  you 
desired . 

4.  The  amount  of  time  5 

between  when  you 

made  the  appt  and 
the  actual  appt  date. 


M  3 

4  3 


4  3 


3 


2 


2 


2 


2 


1 


1 


1 


1 


5.  Information  given 
to  you  by  the  appt 
clerk  with  regard 
to  the  appt. 

6.  Your  overall 
opinion  of  the 
clinic's  appt 
system. 

7.  If  you  can  change 
would  It  be? 


5  4  3 


5  4  3 


anything  about  this  clinic' 


2  1 


2  1 


8  appointment  system,  what 


PLEASE  TURN  TO  PAGE  5  FOR  INSTRUCTIONS  ON  TURNING  IN  YOUR  SURVEY. 
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Other  Appointment  Systems 


If  you  did  not  make  your  most  recent  appointment  by  contacting 
the  Central  Appointment  System  or  by  contacting  the  clinic, 
please  describe  below  how  you  obtained  your  appointment. 


PLEASE  TURN  TO  PAGE  6  FOR  INSTRUCTIONS  ON  TURNING  IN  YOUR  SURVEY. 


Page  4 


84 


Questions  for  those  Indlvlduala 
not  uslnK  the  outpatient  services 
at  HAH C 

1.  The  reason  I  do  not  use  the  outpatient  services  at  MAMC  is: 

[]  MAMC  is  located  too  far  from  my  home  and  is  very 
inconvenient  to  use. (PLEASE  SKIP  REMAINING  QUESTIONS  AND  GO  TO 
PAGE  6  FOR  INSTRUCTIONS) 

[]  It  is  difficult  to  get  an  appointment  at  MAMC.  (GO  TO 
QUESTION  2) 

[]  Other  reason(s}(  Please  specify  your  reason(s)  in  the  space 
below. ) 


GO  TO  PAGE  6  FOR  INSTRUCTIONS. 

2.  My  dissatisfaction  with  the  appointment  system  at  MAMC  is 
related  to  the  following  issues:  (PLEASE  CHECK  ANY  OR  ALL  THE 
ISSUES  THAT  APPLY  TO  YOUR  DISSATISFACTION) 

[]  The  amount  of  time  it  takes  to  make  an  appointment. 

[]  The  way  I’ve  been  treated  by  the  appointment  clerk(s). 

[]  Not  being  able  to  receive  the  specific  appointment  date  or 
time  I  desired. 

[]  The  amount  of  time  between  making  the  appointment  and  the 
actual  appointment  date. 

[]  Information  given  to  me  by  the  appointment  clerk  with 
regard  to  the  appointment. 

C]  The  overall  operation  of  the  appointment  system  at  MAMC. 

3.  If  you  could  change  anything  about  the  current  appointment 
system  at  MAMC  what  would  it  be? 


PLEASE  TURN  TO  PAGE  6  FOR  INSTRUCTIONS. 


Page  5 
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Turn  In  Inatruotlons 


Please  place  your  completed  survey  In  the  attached  self  addressed 
envelope  and  place  It  In  the  mail* 


THAKK  rOD  FOB  YODR  TIME  IN 
COMPLETIMG  THIS  SDRVEY 


Page  6 


APPENDIX  D 

IN-HOSPITAL  SURVEY  AFTER  PRETEST 
(ACTUAL  SURVEY  ADMINISTERED) 


DEAR  PATIENT: 
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A  study  is  being  conducted  to  determine  how  you  feel  about  the  systan  used  for 
making  outpatient  appointments  at  Madigan  Amy  Medical  Center  (MAMC) .  You  are  being 
asked  to  corplete  the  attached  questionnaire  in  order  that  the  system  may  be  more 
responsive  to  your  needs. 

Yoixr  answers  to  the  questions  should  be  based  on  your  experience  making  the  appoint¬ 
ment  for  today's  visit  and  not  on  any  other  experience  you  may  have  had  with  the 
MAMC  Appointment  System.  Your  answers  will  be  combined  with  those  of  other  patients 
and  presented  for  analysis  at  the  ccaipletion  of  the  survey. 

Please  be  open  and  honest  about  your  experience.  Your  answers  cannot  be  connected  with 
you  or  your  physician  since  you  are  not  being  asked  to  provide  specific  information 
about  yourself. 

Your  cooperation  in  completing  this  questionnaire  will  be  greatly  appreciated  and 
will  provide  valuable  information  ^diich  may  be  used  to  make  the  outpatient  aotxaint- 
ment  system  serve  you  better. 


DARRia  H.  POWELL 


Brigadier  General,  Medical  Corps 
Corrmanding 


PLEASE  TURN  TO  THE  NTOT  PAGE  AND  FOLLOW  THE  INSTRUCTIONS  FOR  COMPLETTING  THE 
QUESTIONNAIRE, 
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1.  PLEASE  PLACE  AN  "X"  IN  THE  BOX  IN  FRCMT  OF  THE  STATEMENT  IVHICH  BEST  DESCRIBES 
YOUR  CURRENT  STATUS: 


(  )  Active  Duty  Service  Member 

(  )  Family  Member  of  an  Active  Duty  Service  Manber 
(  )  Retired  Service  Manber 

(  )  Family  Member  of  a  Retired  Service  Manber 
(  )  Family  Member  of  a  Deceased  Service  Member 
(  )  Other 

2,  HCW  LONG  HAVE  YOU  USED  THE  OUTPATIENT  SERVICES  AT  MAMC? 

(  )  Less  than  6  months 
(  )  6  months  to  11  months 
(  )  1  to  3  years 
(  )  More  th^  3  years 

3.  WAT  METTHOD  BEST  DESCRIBES  HCW  YOU  MADE  YOUR  APPOINTi^lENT  FOR  TODAY'S  iTISIT? 


(  )  Called  the  Patient  Appointment  System. 

(  )  Called  this  clinic  and  rtade  the  appointment. 

(  )  Made  the  appointment  in  person  at  this  clinic. 


*IF  YOU  CHOSE  O^IE  OF  THESE 


(  )  Made  the  appointment  by  another  method.  PLEASE  DESCRIBE  Ha-J  YOU  MADE  THE 
APPOINTMENT  IN  THE  SPACE  BELaV. 


AFTER  COMPLETING  THE  DESCRIPTION  OF  HOW  YOU  MADE  THE  APPOINTtlENT  SKIP  THE 
NEXT  PAGE  AND  TURN  TO  PAGE  3  FOR  FURTHER  INSTRUCTIONS.  ^  _ 


PAGE  1 


PAGE  2 
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it'k'kic'kit'kit'kic'kieic-k’k'k'kic’k'k'kicic-k'kic'k'k-k’k'kicit'kicit'k'k’k'k'k'k'k'k'kie’kifkic’k'k’kick'k'k'ki. 

*  ★ 

★INSTRJCTICNS  (PLEASE  READ  CAREFULLY  BEFORE  ANSWERING  ANY  QUESIIC^sl 

*  ★ 

■k************************************************-*!-):***************** 


Please  circle  the  number  v^ich  best  describes  your  feelings  about  each  of  the  following 
issues  related  to  the  way  in  which  you  made  TCDAY'S  APPOE'JTMENT.  Higher  nun±)ers  indicate 
satisfaction  and  lower  numbers  indicate  dissatisfaction.  Please  consider  only  TODAY ' S 
APPOINTMENT  and  not  any  other  time  you  have  made  an  appointment  at  ^lAMC. 


VERY 

SATISFIED 

1.  The  amount  of  5 

time  it  took  you 

to  make  the  appoint¬ 
ment. 

2.  The  way  you  were  5 

treated  by  the 

appointment  clerk. 

3.  Being  able  to  get  5 

your  appointment 

at  the  time  and  date 
you  wanted. 

4.  The  amount  of  time  5 

between  when  you  made 

the  appointment  and 
the  actual  appoint¬ 
ment  date. 

5.  Information  given  to  5 

you  by  the  appoint¬ 
ment  clerk  w:ith  regcird 

to  the  appointment.. 

6.  Your  overall  opinion  5 

of  the  appointment 

system  vou  use  to 
make  the  appointment. 


NEITHER 

SATISFIED/  VEm 

SATISFIED  DISSATISFIED  DISSATISFIED  DISSATISFIED 

4  3  2  1 


4 


3  2  1 


4 


3  2  1 


4  3  2  1 


4  3  2  1 


4 


3  2  1 


7.  If  you  could  change  anything  about  the  appointment  system  you  used  to  make  TODAY'S  APPOINT- 
MENT  what  would  it  be? 


AFTER  COMPLETING  THIS  PAGE  TURN  TO  THE  NEXT  PAGE  FOR  INSTRJCTIONS  FOR  TURNING  IN  THE  SURVEY. 


PAGE  2 
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PAGE  3 


PLEASE  PLACE  YOUR  CX3MPLE1ED  SURVEY  IN  THE  ATTACKED  ENVELOPE,  SEAL  IT,  AND  REIUFN 
IT  TO  THE  CLINIC  RECEPTIC»IIST,  THE  RECEPTKaSfIST  TOLL  NOT  OPEN  THE  SEALED  ENVELOPE 
AND  WILL  INSURE  THAT  THE  SURREY  IS  RETURN  TO  THE  SURVEYOR 


J  'IKANK  YOU  VERY  MUCH  FOR  YOUR  TLME  J 

t  IN  CCMPLETTING  THIS  SURVEY  J 

************************************* 


PAGE  3 


APPENDIX  E 

MAIL-OUT  SURVEY  AFTER  PRETEST 
(ACTUAL  SURVEY  ADMINISTERED) 
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DEAR  RETIREE: 

A  study  is  being  conducted  to  determine  how  you  feel  about  the  system,  used 
for  making  outpatient  appointments  at  fladigan  Army  Medical  Center  (MAMC) . 

You  are  being  asked  to  ccnplete  the  attached  guestionnaire  in  order  that 
the  system  may  be  more  responsive  to  your  needs. 

Your  answers  to  the  Questions  should  be  based  on  your  experiences  makina 
the  appointment  for  YOUR  MOST  RECENT  \rrSIT  TO  MAMC  and  not  on  any  other  ex¬ 
perience  you  may  have  had  with  the  MAfC  Appointment  System.  Your  ans\'«rs 
will  be  ccmbined  with  those  of  other  patients  and  presented  for  analysis  at 
the  corpletion  of  the  survey. 

Please  be  open  and  honest  about  your  experience.  Your  answers  cannot  be 
connected  with  you  or  your  physician  since  you  are  not  being  asked  to 
provide  specific  information  about  yourself. 

Your  cooperation  in  catpleting  this  questionnaire  will  be  greatly  appreciated 
and  will  provide  valuable  information  vdiich  may  be  used  to  make  the  out¬ 
patient  appointment  system  serve  you  better. 


DARRYL*' H.  PCWELL  M.D 


Brigadier  General,  Medical  Corps 
Ccmnanding 


PLEARE  CCHPLETE  THIS  SURVEY  AND  RETURN  IT  IN  THE  ATTACHED  SELF-ADDRESSED 
ENVELOPE  NO  LATER  THAN  20  MAY  1985. 


PLEASE  TURN  TO  THE  NEXT  PAGE  AND  FOUJCM  THE  INSTRUCTIOSIS  FOR  COMPLETING 
EACH  QUESTION. 


PAGE  1 
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1.  PLEASE  PLACE  AN  "X"  IN  THE  SPACE  IN  FRCKr  OF  THE  STATEMOT  WHICH  BEST 
DESCRIBES  YOUR  CURRENT  STATUS: 

(  )  Retired  Service  Manber 

(  )  Fandly  Member  of  a  Retired  Service  Manber 
(  )  Family  Member  of  a  Deceased  Service  Manber 
(  )  Other 

2.  nCM  LOSJG  HAVE  YOU  USED  THE  OUTPATIENT  SERVICES  AT  MAMC? 

************************** 

(  )  I  have  never  used  the  Outpatient  Services  at  MAwMC.*  IF  YOU  CHOOSE  THIS  * 

t  ANSWER-SKIP  TO  PAGE 

*  3  AND  ANSVER  ONLY  THE  J 

*  ouesticns  on  that  page  J 


(  )  Less  than  3  months 
(  )  6  months  to  11  months 
(  )  1  to  3  years 
(  )  More  tlian  3  years 

3.  VJHAT  METHOD  BEST  DESCRIBES  HQ^  YOU  MADE  YOUR  APPOINTMENT  FOR  YOUR  MOST 
RECENT  VISIT  TO  MAMC? 

(  )  called  the  Patient  Appointment  System  f  ff*Yaj*CHOSroNE*orTHESr***L^ 

(  )  Called  the  clinic  and  made  the  appointment  ★  ANSWERS  SKIP  TO  THE  NEXT  PAGeI'^^^ 
(  )  Made  the  cippointment  in  person  at  the  clinic******************************** 

(  )  Made  the  appointment  by  another  method. 

PLEASE  DESCRIBE  HCW  YOU  MADE  THE  APPOINIMENT  IN  THE  SPACE  BELC>7 


AFTER  COMPLETING  THE  DESCRIPTION  OF  H07  YOU  MADE  THE  APPOINTMENT  SKIP 
TO  PAGE  4  FOR  FURTHER  INSTRUCTIONS.  _ _ 


PAGE  1 
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********************************************************************************** 

JINSTPUCTIONS;  (PLEASE  READ  CAREFULLY  KFORE  ANSI^RING  ANY  QUESTIONS  ONI  THIS  PAGE* 
**★**★★****★★**★*★**★******★*★★★*★***★★*★★*****★★★*★**★**★★★★★★*★**★*****★★*★*★**■* 

Please  circle  the  number  ^i^ich  best  describes  your  feelings  about  each  of  the 
following  issues  related  to  the  way  in  viiich  you  made  YCUR  MOST  RECENT  APPOINT¬ 
MENT  AT  MAMC.  Higher  numbers  indicate  satisfaction  and  Icwer  numbers  indicate 
dissatisfaction.  Please  consider  only  YOUR  MOST  RECENT  APPOINTMENT  and  not  any 
other  time  you  have  made  an  appointment  at  MAMC. 

NEITHER 

VERY  SATISFIED/  VERY 

SATISFIED  SATISFIED  DISSATISFIED  DISSATISFIED  DISSATISFIED 

1.  The  amount  of  5  4321 

time  it  took  you 

to  make  the  appt. 

2.  The  way  you  v^re  5  4  3  2  1 

treated  by  the 

appt.  clerk. 

3.  Being  able  to  get  5  4  3  2  1 

your  appt  at  the 

time  and  date  you 
wanted. 

4.  The  amount  of  time  5  4  3  2  1 

between  v^en  you 

made  the  appt  and 
the  actual  appt  date. 

5.  Information  given  to  5  4321 

you  by  the  appt. 

clerk  with  regard 
to  the  appt. 

6.  You  averall  opinion  5  4  3  2  1 

of  the  appt.  system 

you  used  to  make  the 
appt. 


7.  If  you  could  change  anything  about  the  \..^ointment  sysstem  you  used  to  make 
YOUR  MOST  RECENT  APPOINTMENT,  v*iat  WDU^C  it  be? 


AFTER  COMPLETING  THIS  PAGE  TURN  TO  PAGE  4  FOR  INSTRUCTIONS  FOR  RETURNING  YOUR  SURVEY. 

(Ceiir 


PAGE  2 
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PAGE  3 


DO  NOT  ANSWER  THE  QUESTIOSIS  THIS  PAGE  UNLESS  YOU  HAVE  NEVER  USED  THE  OUTPATIENT  SERVICES 
AT  MAMC. 


PLEASE  PLACE  AN  "X"  IN  THE  SPACE  IN  FRCKT  OF  THE  ANSIVER  WHICH  BEST  DESCRIBES  YOUR 
OPINIOJ. 

1.  The  reason  I  do  not  use  the  Outpatient  Services  at  MAMC  is: 

(  )  MAMC  is  located  too  far  from  ray  hare  and  is  very  inconvenient  to  use 

IF  YOU  CHOSE  THIS  ANSV'JER  SKIP  THE  REMAINING  QUESTIONS  AND  GO  TO  THE 
NEXT  PAGE  _ _ _ 

(  )  It  is  difficult  to  get  an  appointment  at  MAMC 

IF  YOU  CHOSE  THIS  ANSI^R  PLEASE  SKIP  TO  QUESTION  2 
(  )  Other  reason (s) 

IF  YOU  CHOSE  THIS  ANSWER  PLEASE  SPECIFY  YOUR  REASONS  IN  THE  SPACE  BELOJ 


AFTER  SPECIFYING  YOUR  REASONS  GO  TO  THE  NEXT  PAGE  FOR  INSTRUCTIONS, 


2.  My  dissatisfaction  with  the  af^intraent  system  at  MAMC  is  related  to  the  following 

issues:  (PLEASE  CHECK  ANY  OR  ALL  OF  THE  ISSUES  THAT  APPLY  TO  YOUR  DISSATISFACTIOSI) 

(  )  The  amount  of  time  it  takes  to  make  an  appointment. 

(  )  The  way  I  have  been  treated  by  the  appointment  clerk (s) . 

(  )  Not  being  able  to  get  the  specific  appointment  date  or  time  I  desired. 

(  )  Information  given  to  me  by  the  appointment  clerk  with  regard  to  the  appointment. 

(  )  The  amount  of  time  between  making  the  appointment  and  the  actiaal  appointment  date. 

(  )  The  overall  operation  of  the  appointment  system  at  MAMC. 

3.  If  could  change  anything  about  the  current  appointment  system  at  MAMC,  what  would 
it  be? 


PLEASE  TORN  TO  THE  NEXT  PAGE  TOR  INSTRUCTIONS. 


PAGE 
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mV  THAT  YOU  HAVE  COMPLETED  THE  SURVEY,  PLEASE  FX)LD  IT  AND  PLACE  IT  IN  THE  SELF-ADDRESSED 
ENVELOPE  AND  PLACE  IT  IN  THE  MAIL.  MI  POSTAGE  IS  REQUIRED  FOR  MATT  .TNG. 

THANK  YOU  VERY  MUCH  FOR  YOUR  TIME  IN  COMPLETING  THIS  SURVEY! !!!!!! 


PAGE  4 
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